You are a head of BI at a large enterprise company with multiple business units that each have different priorities and budgets. You use on-demand pricing for BigQuery with a quota of 2K concurrent on-demand slots per project. Users at your organization sometimes don't get slots to execute their query and you need to correct this. You'd like to avoid introducing new projects to your account.What should you do?
Answer(s): C
https://cloud.google.com/blog/products/gcp/busting-12-myths-about-bigquery
You have an Apache Kafka Cluster on-prem with topics containing web application logs. You need to replicate the data to Google Cloud for analysis in BigQuery and Cloud Storage. The preferred replication method is mirroring to avoid deployment of Kafka Connect plugins.What should you do?
Answer(s): A
You've migrated a Hadoop job from an on-prem cluster to dataproc and GCS. Your Spark job is a complicated analytical workload that consists of many shuffing operations and initial data are parquet files (on average 200-400 MB size each). You see some degradation in performance after the migration to Dataproc, so you'd like to optimize for it. You need to keep in mind that your organization is very cost-sensitive, so you'd like to continue using Dataproc on preemptibles (with 2 non-preemptible workers only) for this workload.What should you do?
To optimize the performance of a complex Spark job on Dataproc that heavily relies on shuffling operations, and given the cost constraints of using preemptible VMs, switching from HDDs to SSDs and using HDFS as an intermediate storage layer can significantly improve performance. Here's why option C is the best choice:Performance of SSDs:SSDs provide much faster read and write speeds compared to HDDs, which is crucial for performance-intensive operations like shuffling in Spark jobs.Using SSDs can reduce I/O bottlenecks during the shuffle phase of your Spark job, improving overall job performance.Intermediate Storage with HDFS:Copying data from Google Cloud Storage (GCS) to HDFS for intermediate storage can reduce latency compared to reading directly from GCS.HDFS provides better locality and faster data access within the Dataproc cluster, which can significantly improve the efficiency of shuffling and other I/O operations.Cost Considerations:Although SSDs are more expensive than HDDs, the performance improvement for shuffle-heavy workloads can justify the cost, especially if the improved performance reduces the overall runtime and thereby the cost of using preemptible VMs.Using preemptible VMs with SSDs for this workload balances the cost and performance trade-offs effectively.
Your team is responsible for developing and maintaining ETLs in your company. One of your Dataflow jobs is failing because of some errors in the input data, and you need to improve reliability of the pipeline (incl. being able to reprocess all failing data).What should you do?
You're training a model to predict housing prices based on an available dataset with real estate properties. Your plan is to train a fully connected neural net, and you've discovered that the dataset contains latitude and longtitude of the property. Real estate professionals have told you that the location of the property is highly influential on price, so you'd like to engineer a feature that incorporates this physical dependency.What should you do?
Answer(s): B
Feature Crosses:Feature crosses combine multiple features into a single feature that captures the interaction between them. For location data, a feature cross of latitude and longitude can capture spatial dependencies that affect housing prices.This approach allows the neural network to learn complex patterns related to geographic location more effectively than using raw latitude and longitude values.Numerical Representation:Converting the feature cross into a numeric column simplifies the input for the neural network and can improve the model's ability to learn from the data.This method ensures that the model can leverage the combined information from both latitude and longitude in a meaningful way.Model Training:Using a numeric column for the feature cross helps in regularizing the model and prevents overfitting, which is crucial for achieving good generalization on unseen data.
https://cloud.google.com/bigquery/docs/gis-dataaTo engineer a feature that incorporates the physical dependency of location on housing prices for a neural network, creating a numeric column from a feature cross of latitude and longitude is the most effective approach. Here's why option B is the best choice:
Post your Comments and Discuss Google Professional Data Engineer exam prep with other Community members:
Our website is free, but we have to fight against AI bots and content theft. We're sorry for the inconvenience caused by these security measures. You can access the rest of the Professional Data Engineer content, but please register or login to continue.