Google BigQuery updates: Faster, easier and more data formats

Posted by Ju-kay Kwek, Product Manager

We know you have a lot of data to work with within your organization, which can present big challenges. Your data can be large in volume and complex in structure. For example, large-scale web applications have millions of users, documents and events to manage. As a result, many engineering teams choose highly scalable NoSQL databases over relational databases. Though this approach is effective in storing and retrieving data, it poses challenges for interactive data analysis.

Today’s release of Google BigQuery tackles these hurdles with several new features:

• Support for JSON: JSON is used to power most modern websites, is a native format for many NoSQL databases hosting large scale web applications, and is used as the primary data format in many REST APIs. With this update, it’s now possible to import data formatted in JSON directly to BigQuery without the hassle of writing extra code to convert the data format.
  
  
• Nested and Repeated Fields: If you’re using App Engine Datastore or other NoSQL databases, it’s likely you’re taking advantage of nested and repeated data in your data model. For example, a customer data entity might have multiple accounts, each storing a list of invoices. Now, instead of having to flatten that data, you can keep your data in a hierarchical format when you import to BigQuery.
   
• Additional improvements: 
• Increased import quotas from 1000 jobs per day to 1000 jobs per table per day, and boosted the file size limit from 4GB to 100GB 
• Faster data exports from BigQuery to Google Cloud Storage, by enabling large tables to be exported as multiple files in parallel 
• Permanently save common queries in the BigQuery interface 

To learn more about how Google BigQuery can help you gain insights from your data in the cloud, click here to sign up.