HTTP Archive problems: one request to Google BigQuery cost $14,000

Teacher

Professional
Messages
2,673
Reputation
9
Reaction score
682
Points
113
The incident caused a wide discussion in the community and a change in the service's website.

In the cloud industry, a discussion has flared up about how providers should limit the use of their tools after a Google BigQuery user was billed thousands of dollars for simply executing queries.

A user of the HTTP Archive project, whose task is to analyze the structure of the web, received a bill for $14,000 from Google. A user under the nickname Tim expressed outrage on the HTTP Archive forum, saying that the data set presented as publicly available is actually a source of profit for Google Cloud.

Tim called for a warning to be added to the official website about the commercial use of Google data, noting the lack of customer support and the risk of losing $14,000 dollars "in a second". In response, the archive administrator said that 99% of archive users use only free monthly and annual Web Almanac reports, and BigQuery is intended for 1% of the most demanding users who need more detailed access to data.

A representative of HTTP Archive clarified that the $14,000 bill corresponds to processing approximately 2.5 Pbytes (petabytes) of data at a price of $6.25 per TiB (tebibyte), and promised to add a clearer warning about BigQuery rates to the site's FAQ page.

Tim also pointed out that there is no mechanism for warning about the cost of a request in the official GCP libraries for Python, as opposed to the web interface. He suggested introducing an automatic spending cap of $5,000 to prevent requests from being executed without user confirmation.

During the discussion, one of the participants, whose comment was hidden by the moderators, called Tim unwise for launching a request without understanding the amount of data being processed. Despite the fact that Google provides full information about the cost of BigQuery services on its website, students and academics may face unexpected high bills due to a lack of understanding of the terms.
 
Top