Feature engineering - quantifying access to facilities¶
Often, when shortlisting facilities buyers look for access to facilities such as groceries, restaurants, schools, emergency and health care in thier neighborhood. In this notebook, we use the geocoding
module to search for such facilities and build a table for each property.
import pandas as pd
import matplotlib.pyplot as plt
from pprint import pprint
%matplotlib inline
from arcgis.gis import GIS
from arcgis.geocoding import geocode
from arcgis.features import Feature, FeatureLayer, FeatureSet, GeoAccessor, GeoSeriesAccessor
from arcgis.features import SpatialDataFrame
from arcgis.geometry import Geometry, Point
from arcgis.geometry.functions import buffer
from arcgis.network import RouteLayer
Connect to GIS
gis = GIS(profile='')
Read one of the shortlisted properties¶
prop_list_df = pd.read_csv('resources/houses_for_sale_att_filtered.csv')
prop_list_df.shape
(331, 24)
prop_list_df = pd.DataFrame.spatial.from_xy(prop_list_df, 'LONGITUDE','LATITUDE')
type(prop_list_df)
pandas.core.frame.DataFrame
prop1 = prop_list_df[prop_list_df['MLS']==18389440]
prop1
Unnamed: 0 | SALE TYPE | PROPERTY TYPE | ADDRESS | CITY | STATE | ZIP | PRICE | BEDS | BATHS | ... | DAYS ON MARKET | PRICE PER SQFT | HOA PER MONTH | STATUS | URL | SOURCE | MLS | LATITUDE | LONGITUDE | SHAPE | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
203 | 2328 | MLS Listing | Single Family Residential | 3775 NW Hilton Head Ter | Portland | OR | 97229.0 | 649900.0 | 4.0 | 2.5 | ... | 21.0 | 221.0 | 25.0 | Active | http://www.redfin.com/OR/Portland/3775-NW-Hilt... | RMLS | 18389440 | 45.546644 | -122.815658 | {"x": -122.8156584, "y": 45.546644, "spatialRe... |
1 rows × 24 columns
Create symbols for facilities¶
Get your symbols using this online tool: http://esri.github.io/arcgis-python-api/tools/symbol.html
house_symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/Shapes/RedStarLargeB.png","contentType":"image/png","width":24,"height":24}
grocery_symbol = symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/PeoplePlaces/Shopping.png","contentType":"image/png","width":12,"height":12}
hospital_symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/SafetyHealth/Hospital.png","contentType":"image/png","width":24,"height":24}
coffee_symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/PeoplePlaces/Coffee.png","contentType":"image/png","width":12,"height":12}
restaurant_symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/PeoplePlaces/Dining.png","contentType":"image/png","width":12,"height":12}
bar_symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/PeoplePlaces/Bar.png","contentType":"image/png","width":12,"height":12}
gas_symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/Transportation/esriBusinessMarker_72.png","contentType":"image/png","width":12,"height":12}
shops_service_symbol={"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/PeoplePlaces/esriBusinessMarker_58_Red.png","contentType":"image/png","width":10,"height":10}
transport_symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/Transportation/esriDefaultMarker_195_White.png","contentType":"image/png","width":15,"height":15}
professional_symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/PeoplePlaces/esriBusinessMarker_64_Yellow.png","contentType":"image/png","width":10,"height":10}
parks_symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/OutdoorRecreation/RestArea.png","contentType":"image/png","width":10,"height":10}
education_symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/PeoplePlaces/Note.png","contentType":"image/png","width":10,"height":10}
arts_symbol = {"angle":0,"xoffset":0,"yoffset":0,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/PeoplePlaces/LiveShow.png","contentType":"image/png","width":12,"height":12}
destination_symbol = {"angle":0,"xoffset":0,"yoffset":12,"type":"esriPMS","url":"http://static.arcgis.com/images/Symbols/Basic/RedStickpin.png","contentType":"image/png","width":24,"height":24}
fill_symbol = {"type": "esriSFS","style": "esriSFSNull",
"outline":{"color": [255,0,0,255]}}
fill_symbol2 = {"type": "esriSFS","style": "esriSFSNull",
"outline":{"color": [0,0,0,255]}}
route_symbol = {"type": "esriSLS","style": "esriSLSSolid",
"color": [0, 120, 255, 255],"width": 1.5}
Get 5 mile extent around the property of interest¶
paddress = prop1.ADDRESS + ", " + prop1.CITY + ", " + prop1.STATE
prop_geom_fset = geocode(paddress.values[0], as_featureset=True)
Create an envelope around the property using its extent
prop_geom = prop_geom_fset.features[0]
prop_geom.geometry
{'x': -122.81532305026238, 'y': 45.54674878544642, 'spatialReference': {'wkid': 4326, 'latestWkid': 4326}}
prop_geom = prop_geom_fset.features[0]
prop_buffer = buffer([prop_geom.geometry],
in_sr = 102100, buffer_sr=102100,
distances=0.05, unit=9001)[0]
prop_buffer_f = Feature(geometry=prop_buffer)
prop_buffer_fset = FeatureSet([prop_buffer_f])
Plot house and buffer on map¶
pdx_map = gis.map('Portland, OR')
pdx_map.basemap='gray'
pdx_map
MapView(basemap='gray', layout=Layout(height='400px', width='100%'))
pdx_map.draw(prop_buffer_fset, symbol=fill_symbol2)
pdx_map.draw(prop_geom_fset, symbol=house_symbol)
neighborhood_data_dict = {}
groceries = geocode('groceries', search_extent=prop_buffer.extent,
max_locations=20, as_featureset=True)
neighborhood_data_dict['groceries'] = []
for place in groceries:
popup={"title" : place.attributes['PlaceName'],
"content" : place.attributes['Place_addr']}
pdx_map.draw(place.geometry, symbol=grocery_symbol, popup=popup)
neighborhood_data_dict['groceries'].append(place.attributes['PlaceName'])
We will geocode for the following facilities within the said 5
mile buffer.
Groceries
Restaurants
Hospitals
Coffee shops
Bars
Gas stations
Shops and service
Travel and transport
Parks and outdoors
Education
Restaurants¶
pdx_map2 = gis.map('Portland, OR')
pdx_map2.basemap='gray'
pdx_map2
MapView(basemap='gray', layout=Layout(height='400px', width='100%'))
pdx_map2.draw(prop_buffer_fset, symbol=fill_symbol2)
pdx_map2.draw(prop_geom_fset, symbol=house_symbol)
restaurants = geocode('restaurant', search_extent=prop_buffer.extent, max_locations=200)
neighborhood_data_dict['restauruants'] = []
for place in restaurants:
popup={"title" : place['attributes']['PlaceName'],
"content" : place['attributes']['Place_addr']}
pdx_map2.draw(place['location'], symbol=restaurant_symbol, popup=popup)
neighborhood_data_dict['restauruants'].append(place['attributes']['PlaceName'])
Hospitals¶
hospitals = geocode('hospital', search_extent=prop_buffer.extent, max_locations=50)
neighborhood_data_dict['hospitals'] = []
for place in hospitals:
popup={"title" : place['attributes']['PlaceName'],
"content" : place['attributes']['Place_addr']}
pdx_map2.draw(place['location'], symbol=hospital_symbol, popup=popup)
neighborhood_data_dict['hospitals'].append(place['attributes']['PlaceName'])
Coffee shops¶
coffees = geocode('coffee', search_extent=prop_buffer.extent, max_locations=50)
neighborhood_data_dict['coffees'] = []
for place in coffees:
popup={"title" : place['attributes']['PlaceName'],
"content" : place['attributes']['Place_addr']}
pdx_map2.draw(place['location'], symbol=coffee_symbol, popup=popup)
neighborhood_data_dict['coffees'].append(place['attributes']['PlaceName'])
Bars¶
bars = geocode('bar', search_extent=prop_buffer.extent, max_locations=50)
neighborhood_data_dict['bars'] = []
for place in bars:
popup={"title" : place['attributes']['PlaceName'],
"content" : place['attributes']['Place_addr']}
pdx_map2.draw(place['location'], symbol=bar_symbol, popup=popup)
neighborhood_data_dict['bars'].append(place['attributes']['PlaceName'])
Gas stations¶
gas = geocode('gas station', search_extent=prop_buffer.extent, max_locations=50)
neighborhood_data_dict['gas'] = []
for place in gas:
popup={"title" : place['attributes']['PlaceName'],
"content" : place['attributes']['Place_addr']}
pdx_map2.draw(place['location'], symbol=gas_symbol, popup=popup)
neighborhood_data_dict['gas'].append(place['attributes']['PlaceName'])
Shops and service¶
shops_service = geocode("",category='shops and service', search_extent=prop_buffer.extent, max_locations=50)
neighborhood_data_dict['shops'] = []
for place in shops_service:
popup={"title" : place['attributes']['PlaceName'],
"content" : place['attributes']['Place_addr']}
pdx_map2.draw(place['location'], symbol=shops_service_symbol, popup=popup)
neighborhood_data_dict['shops'].append(place['attributes']['PlaceName'])
Travel and transport¶
transport = geocode("",category='travel and transport', search_extent=prop_buffer.extent, max_locations=50)
neighborhood_data_dict['transport'] = []
for place in transport:
popup={"title" : place['attributes']['PlaceName'],
"content" : place['attributes']['Place_addr']}
pdx_map2.draw(place['location'], symbol=transport_symbol, popup=popup)
neighborhood_data_dict['transport'].append(place['attributes']['PlaceName'])
Parks and outdoors¶
parks = geocode("",category='parks and outdoors', search_extent=prop_buffer.extent, max_locations=50)
neighborhood_data_dict['parks'] = []
for place in parks:
popup={"title" : place['attributes']['PlaceName'],
"content" : place['attributes']['Place_addr']}
pdx_map2.draw(place['location'], symbol=parks_symbol, popup=popup)
neighborhood_data_dict['parks'].append(place['attributes']['PlaceName'])
Education¶
education = geocode("",category='education', search_extent=prop_buffer.extent, max_locations=50)
neighborhood_data_dict['education'] = []
for place in education:
popup={"title" : place['attributes']['PlaceName'],
"content" : place['attributes']['Place_addr']}
pdx_map2.draw(place['location'], symbol=education_symbol, popup=popup)
neighborhood_data_dict['education'].append(place['attributes']['PlaceName'])
Present the results in a table¶
neighborhood_df = pd.DataFrame.from_dict(neighborhood_data_dict, orient='index')
neighborhood_df = neighborhood_df.transpose()
neighborhood_df
groceries | restauruants | hospitals | coffees | bars | gas | shops | transport | parks | education | |
---|---|---|---|---|---|---|---|---|---|---|
0 | Bales Market Place | Coffee. Cup | Providence St Vincent Medical Center-ER | Coffee. Cup | None | Shell | Anderson Towing & Recovery | MAX-Elmonica & SW 170th Ave | Jqay House Park | Portland Community College-Rock Creek |
1 | Safeway | Papa Murphy's | Providence St Vincent Medical Center | Starbucks | None | ARCO | Bassitt Auto Co | Powder Lodging | Jackie Husen Park | Cedar Mill Elementary School |
2 | QFC | Tilly's Gelato | None | Starbucks | None | 76 | Cottman Transmission | Homestead Studio Suites-Beaverton | The Bluffs | Goddard School |
3 | Dinihanian's Farm Market | Oak Hills Brew Pub | None | Poppa's Haven | None | Costco | Powell Paint Center | MAX-Sunset TC | Bonny Slope Park | St Pius X Elementary School |
4 | Walmart Neighborhood Market | Starbucks | None | Tazza Cafe | None | 76 | Retied | MAX-Merlo & SW 158th Ave | Burton Park | Terra Linda Elementary School |
5 | India Supermarket | Starbucks | None | Laughing Planet Cafe | None | Fred Meyer Fuel Center | Chrisman's Picture Frame & Gallery | MAX-Beaverton Creek | Lost Park | Montessori School of Beaverton |
6 | Apna Bazaar | Chiam | None | Coffee Renaissance | None | Chevron | Team Uniforms | Rodeway Inn & Suites-Portland | Terra Linda Park | A Childs Way Kindergarten |
7 | Fred Meyer | Bandito Taco | None | Starbucks | None | Chevron | T-Mobile | Doubletree-Beaverton | Jordan Park | Christ United Methodist Preschool |
8 | Albertsons | Chipotle | None | Bowl & Berry | None | 76 | Holistic Pet | Hilton Garden Inn-Beaverton | Cedar Mill Woods Park | Cornell Children's Village Day Sch |
9 | WinCo Foods | Bollywood Bites | None | Taiwan Eats | None | Shell | Shell | Fairfield Inn & Suites-Beaverton | Cedar Mill Park | Catlin Gabel School |
10 | Plaid Pantry | Pizza Schmizza | None | Starbucks | None | 76 | Mike's Auto Parts | Homewood Suites-Hillsboro Beaverton | Roger Tilbury Memorial Park | West Tualatin View Elementary Sch |
11 | Target | Wan Q Restaurant | None | Bethany Public House | None | Shell | Kaady Car Wash | Poehler Airport | Merritt Orchard Park | Beaver Acres Elementary School |
12 | None | Shari's | None | Starbucks | None | 76 | 7-Eleven | Park & Ride-Elmonica/SW 170th Ave | Commonwealth Park | Ridgewood Elementary School |
13 | None | Ct Bistro | None | Starbucks | None | None | Motor Sports International | Park & Ride-NW Cornell | Pioneer Park | William Walker Elementary School |
14 | None | McDonald's | None | Creatures of Habit Espresso & Deli | None | None | Bank of America | Park & Ride-Sunset TRANSIT Center | Peppertree Park | Meadow Park Middle School |
15 | None | Pizzicato Westside | None | Starbucks | None | None | Bales Market Place | Park & Ride-Beaverton Creek | Claremont Golf Course | Pasitos Spanish School |
16 | None | Lejoi Cafe | None | Coffee Rush | None | None | US Bank | Park & Ride-Cedar Hills Church | Kaiser Woods Park | Holy Trinity Elementary School |
17 | None | Ichiban Japanese | None | Starbucks | None | None | Dufresne's Auto Service | Dock | Bethany Meadows Park | Barnes Elementary School |
18 | None | Mac's Market & Deli | None | None | None | None | US Bank | Dock | Kaiser Ridge Park | Shiny Sparkles Montessori |
19 | None | SUBWAY | None | None | None | None | Bank of America | Fred Meyer | Morgans Run Park | Stoller Middle School |
20 | None | Salars Mediterranean Grill | None | None | None | None | Ace Hardware | Dock | West Union Estates Park | Findley Elementary |
21 | None | Poppa's Haven | None | None | None | None | Du Fresne's Auto Repair | Dock | College Park | Kindercare Learning Center |
22 | None | Papa John's | None | None | None | None | Jiffy Lube | Dock | Northeast Neighborhood Park | Westview Senior High School |
23 | None | Tazza Cafe | None | None | None | None | Wells Fargo | Dock | Ben Graf Park | Beaverton School |
24 | None | Teriyaki Beef Bowl | None | None | None | None | Walgreens | 1-1 | Spyglass Park | Sweet Peas Kidzone |
25 | None | China Rim | None | None | None | None | Wells Fargo | Exit 67/Murray Blvd/E | Stoller Farms Park | La Petite Academy |
26 | None | SUBWAY | None | None | None | None | Safeway | Exit 65/Bethany Blvd/E | Emerald Estates Park | Oak Hills Elementary School |
27 | None | Dairy Queen | None | None | None | None | Team Uniforms | Exit 67/Murray Blvd/W | Serah Lindsay Estates Park | Bethany Elementary School |
28 | None | Si Senor Mexican Restaurant | None | None | None | None | Xpressolube | Exit 69B/Park Way/W | Quarry Park | Kids of the Kingdom |
29 | None | Laughing Planet Cafe | None | None | None | None | Mike's Auto Parts | Exit 68/Cedar Hills Blvd/W | Springville Meadows Park | Elmonica Elementary School |
30 | None | Pizza Schmizza | None | None | None | None | SHERWIN-WILLIAMS | Tillamook/N | Skyview Park | Touchstone School |
31 | None | Bleachers | None | None | None | None | Pet Barn | Exit 65/Cornell Rd/W | John Marty Park | Five Oaks Middle School |
32 | None | Tazza Cafe | None | None | None | None | Rock It Resell | Exit 69B/Cedar Hills/E | George W Otten Park | Pacific Academy |
33 | None | Bowl & Berry | None | None | None | None | Baby & Me | Exit 68/Cedar Hills Blvd/E | Somerset Meadows Park | Merlo Station High School |
34 | None | Sweet Lemon Vegetarian Bistro | None | None | None | None | Cedar Mill Home Theater | Exit 1/Walker Rd/S | Oak Hills Park | Kinder Prep Private Preschool |
35 | None | Juan Colorado | None | None | None | None | Dollar Tree | Exit 64/185th Ave/W | Bronson Creek Park | Prince of Peace Lutheran Preschool |
36 | None | Taiwan Eats | None | None | None | None | Sunset Science Park Federal CU | None | Autumn Ridge Park | Sunset High School |
37 | None | Taiwan Eats | None | None | None | None | Sunset Science Park Federal CU | None | Apollo Ridge Park | Cedar Hills Kindercare |
38 | None | Cackalack's Hot Chicken Shack | None | None | None | None | Dennis Market | None | Willow Creek Nature Park | Northwest Montessori School |
39 | None | Chen's Dynasty Restaurant | None | None | None | None | Laughing Planet Cafe | None | Dwight S Parr Jr Park | Agia Sophia Academy |
40 | None | Bethany Sushi | None | None | None | None | None | None | Moshofsky Woods Park | Learning Years Day School |
41 | None | SUBWAY | None | None | None | None | None | None | Stonegate at Willow Creek | ABC Children's Academy |
42 | None | Starbucks | None | None | None | None | None | None | Waterhouse Park | Arco Iris Spanish Immersion School |
43 | None | Bethany Public House | None | None | None | None | None | None | Sunset Park | Forest Park Elementary School |
44 | None | Bethany Public House | None | None | None | None | None | None | Foege Park | None |
45 | None | Bliss Bake Shoppe | None | None | None | None | None | None | Peterkort Village Park | None |
46 | None | Bliss Bake Shoppe | None | None | None | None | None | None | Wanda L Peck Memorial Park | None |
47 | None | Biscuits Cafe | None | None | None | None | None | None | Howard M Terpenning Complex | None |
48 | None | Koi Fusion | None | None | None | None | None | None | Forest Heights Park | None |
49 | None | Starbucks | None | None | None | None | None | None | Forest Heights City Park | None |
neighborhood_df.count().plot(kind='bar')
plt.title('Facilities within 5 miles of {}'.format(prop1.ADDRESS.values[0]))
Text(0.5,1,'Facilities within 5 miles of 3775 NW Hilton Head Ter')
Find duration to commute to work¶
route_service_url = gis.properties.helperServices.route.url
route_service = RouteLayer(route_service_url, gis=gis)
stops = [paddress.values[0], '309 SW 6th Ave #600, Portland, OR 97204']
from arcgis.geocoding import geocode, batch_geocode
stops_geocoded = batch_geocode(stops)
stops_geocoded = [item['location'] for item in stops_geocoded]
stops_geocoded2 = '{},{};{},{}'.format(stops_geocoded[0]['x'],stops_geocoded[0]['y'],
stops_geocoded[1]['x'],stops_geocoded[1]['y'])
stops_geocoded2
'-122.81532304999996,45.546748785000034;-122.67727209699996,45.52153932300007'
modes = route_service.retrieve_travel_modes()['supportedTravelModes']
for mode in modes:
print(mode['name'])
Walking Time Rural Driving Distance Driving Time Driving Distance Walking Distance Rural Driving Time Trucking Time Trucking Distance
route_service.properties.impedance
'TravelTime'
Calculate time it takes to get to work. Set start time as 8:00 AM
on Mondays. ArcGIS routing service will use historic averages, so we provide this time as 8:00 AM, Monday, June 4 1990
in Unix epoch time. Read more about this here
route_result = route_service.solve(stops_geocoded2, return_routes=True,
return_stops=True, return_directions=True,
impedance_attribute_name='TravelTime',
start_time=644511600000,
return_barriers=False, return_polygon_barriers=False,
return_polyline_barriers=False)
route_length = route_result['directions'][0]['summary']['totalLength']
route_duration = route_result['directions'][0]['summary']['totalTime']
route_duration_str = "{}m, {}s".format(int(route_duration),
round((route_duration %1)*60,2))
print("route length: {} miles, route duration: {}".format(round(route_length,3),
route_duration_str))
route length: 10.273 miles, route duration: 27m, 48.39s
route_features = route_result['routes']['features']
route_fset = FeatureSet(route_features)
stop_features = route_result['stops']['features']
stop_fset = FeatureSet(stop_features)
route_pop_up = {'title':'Name',
'content':'Total_Miles'}
pdx_map2.draw(route_fset, symbol=route_symbol, popup=route_pop_up)
pdx_map2.draw(stop_fset, symbol=destination_symbol)