http://www.unbounddna.com Fri, 24 Apr 2026 09:00:00 +0000 en-US hourly 1 https://wordpress.org/?v=3.9.40 https://craigsmith.id.au/2017/05/05/google-cloud-data-pipeline-patterns/ https://craigsmith.id.au/2017/05/05/google-cloud-data-pipeline-patterns/#comments Fri, 05 May 2017 06:13:39 +0000 http://craigsmith.id.au/?p=2190 Continue reading Google Cloud Data Pipeline Patterns]]> At the recent YOW! Night in Brisbane (as well and Sydney and Melbourne), Lynn Langit, and independent software consultant and cloud expert presented “Google Cloud Data Pipeline Patterns”.  It was great to meet her, here are my notes from the event:

• Storage is so simple you don’t need to think about it – data lake architecture
• Simply select how much memory and how many cores and it computes the cost
• Google is quick because they are laying their own fibre and using their own infrastructure – less than a minute to spin up a VM, with no local data centre
• Chrome has a RDP client – who knew
• Google philosophy is you shouldn’t have to hire someone to manage the pricing
• Bioinformatics is new to the cloud and they are in great need of engineers, Google is having a stab at a genomics API
• BigQuery is NoOps, has been evolved to be a data warehouse, with no servers,
• Big Relational if you just want SQL, Amazon Aurora is the fastest growing cloud database, Google has released Cloud Spanner, first database to truly meet CAP theory
• BigQuery is column store, Cloud Spanner is OLTP
• MQTT seems to be coming the defacto standard for IoT
• IoT uses BigTable first and BigQuery second in Google reference architectures, because it is cheaper
• Python is the emerging ML language for some reason
• Teaching Kids Programming

]]> https://craigsmith.id.au/2017/05/05/google-cloud-data-pipeline-patterns/feed/ 0