When analyzing Cloud Storage use, we consider three needs:
Setting a lifecycle policy lets you tag specific objects or buckets and creates an automatic rule that will delete or even transform storage classes for that particular object or bucket type.
The ability to transform objects into lower-cost storage classes is a powerful tool, but one that must be used with caution. …
gcloud dataproc clusters create my-pinned-cluster --image-version 1.4-debian9
_PARTITIONTIMEpseudo column to filter the partitions.
BigQuery operations that are free of charge in any location:
For any location, the BigQuery pricing is broken down like this:
2 Query processing
This post looks at calling BigQuery from devices through REST API. Devices often have a small footprint and have constraints and may not be possible to install Client Libraries. By using JWT directly as a bearer token, rather than an OAuth 2.0 access token, you can avoid having to make a network request to Google’s authorization server before making an API call.
BigQuery provides an excellent CLI — “bq extract” and an API as well as Client Library in many languages for achieving the same. In exporting table data documentation it says — You can use a service such as Dataflow to read data from BigQuery instead of manually exporting it. This post looks at the Dataflow way to extract data out of BigQuery. This is useful in situations where “bq extract” doesn’t meet requirements and you really need a programmatic way to extract and manipulate into files.
As an example, I have used the Chicago Taxi Trips public dataset and the…
Google Cloud enthusiast