Big Data In The Browser

So why would anyone want to put alot of data into a browser? Well, for a lot of the same reasons that edge computing and distributed computing have become so popular.

You get the data a lot closer to the user and you don’t have to pay for the compute ;)

… this sounds great but as I found out during this conversation it's not as easy as it might seem!

There are a lot of trade-offs that need to be evaluated when moving data and analytics to the client.

Nick Rabinowitz (linkedin.com/in/nrabinowitz) Senior Staff Software Engineer at Foursquare has a ton of experience with this so he volunteered his time to help us understand more about it.

location.foursquare.com

studio.foursquare.com/home

If you are not familiar with the Arrow data format (arrow.apache.org) it might be worth checking out

Apache Arrow defines a language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware like CPUs and GPUs. The Arrow memory format also supports zero-copy reads for lightning-fast data access without serialization overhead

Related podcast episodes that you might find interesting include

H3 grid system

mapscaping.com/podcast/h3-geospatial-indexing-…

The H3 geospatial indexing system is a discrete global grid system consisting of a multi-precision hexagonal tiling of the sphere with hierarchical indexes. H3 is a really interesting approach to tiling data that was developed by UBER and has been open-sourced.

Hex Tiles

mapscaping.com/podcast/hex-tiles

If you have not heard of the H3 grid system before listen to that episode first before listening to this one it will add a lot of useful context!

Spatial Knowledge Graphs

mapscaping.com/podcast/spatial-knowledge-graphs

Foursquare is moving away from spatial joins and focusing on building a knowledge graph. If you are not familiar with graphs this might be a good place to start, also its interesting to hear the reasons for the move from spatial joins to another data structure.

Distribution Geospatial Data

mapscaping.com/podcast/distributing-geospatial…

This is interesting if you want to understand more about distributed databases and some of the strategies for doing this. It sounds complicated but this episode is a really good introduction!

Cloud Native Geospatial

mapscaping.com/podcast/cloud-native-geospatial

This episode give a solid overview of what cloud-native means and some of the current geospatial cloud native formats out there today

I am constantly thinking about how I can make this podcast better for you so if you have any ideas or suggestions please let me know!

Also, I am thinking of recording a behind-the-scenes episode, is that something you might be interested in? if so what questions do you have?

Some more episodes you might enjoy

ESRI (mapscaping.podbean.com), GIS careers (mapscaping.com/podcasts), Geospatial Data Science (mapscaping.com/podcasts)

QGIS (mapscaping.com/podcasts), Geospatial Python (mapscaping.com/podcasts), ArcGIS Pro (mapscaping.com/podcasts)

Google Maps (mapscaping.com/podcasts), Geomatics (mapscaping.com/podcasts), Cartography (mapscaping.com/podcasts)

Location Intelligence (mapscaping.com/podcasts), Mapping (mapscaping.com/podcasts)