Frank Contrepois

GCP Professional Cloud Architect study

Posted:   |   Categories: tech

I am studying to pass the GCP exam and a very confusing part of all Cloud Providers is the amount of products available and how the names are confusing. So here is a list of GCP services and a small summary of what they are.

GCP for AWS experts https://cloud.google.com/docs/compare/aws/

This is a WIP and is not planned to be the ultimate guide of anything, that’s what official documentation is for.

I recommend to read (and understand) each architectures of [https://gcp.solutions/], my favorite are listed at the end of the post.

  • AppEngine: is a platform for building scalable web applications and mobile and IoT backends. App Engine provides you with built-in services and APIs, such as NoSQL datastores, memcache, and a user authentication API, common to most applications. App Engine will scale your application automatically in response to the amount of traffic it receives, so you only pay for the resources you use. [https://cloud.google.com/appengine/docs/the-appengine-environments]
  • AppEngine standard environment: Applications that need to deal with rapid scaling. All is managed by gcp
  • AppEngine flexible environment: Application instances run within Docker containers on Compute Engine virtual machines (VM).
  • BigQuery: BigQuery is Google’s fully managed, low cost, serverless data warehouse that scales with your storage and computing power needs. With BigQuery, you get a columnar and ANSI SQL database that can analyze terabytes to petabytes of data in blazing-fast speeds. Can capture traces from all of your VMs, containers, or App Engine projects. Only full scan, no indexes. BigQuery charges for data storage, streaming inserts, and for querying data, but loading and exporting data are free of charge. Storage that automatically gets cheaper as your data become less frequently accessed allows you to keep long-term events for training machine learning models on Cloud ML. Analytics backend, DataLake [https://gcp.solutions/diagram/db-backend-database] [https://cloud.google.com/bigquery/]
  • Cloud Bigtable: provides a massively scalable NoSQL database suitable for low-latency and high-throughput workloads. It integrates easily with popular big-data tools like Hadoop and Spark, and it supports the open-source, industry-standard HBase API. Time series, Event Logging. [https://gcp.solutions/diagram/db-backend-database]
  • Cloud Build: Cloud Build lets you build software quickly across all languages. Get complete control over defining custom workflows for building, testing, and deploying across multiple environments such as VMs, serverless, Kubernetes, or Firebase. [https://cloud.google.com/cloud-build/]
  • Cloud Composer: is a fully managed workflow orchestration service that empowers you to author, schedule, and monitor pipelines that span across clouds and on-premises data centers. Built on the popular Apache Airflow open source project and operated using the Python programming language, Cloud Composer is free from lock-in and easy to use.
  • Cloud Data Fusion: is a fully managed, cloud-native data integration service that helps users efficiently build and manage ETL/ELT data pipelines.
  • Cloud Data Lose prevention: Cloud DLP helps you better understand and manage sensitive data. It provides fast, scalable classification and redaction for sensitive data elements like credit card numbers, names, social security numbers, US and selected international identifier numbers, phone numbers, and GCP credentials.
  • Cloud Data Transfer: offers solutions that meet your unique data transfer needs and move data to Cloud Storage, BigQuery, or Cloud Dataproc quickly and securely.
  • Cloud Dataflow: is a fully managed service for transforming and enriching data in stream (real time) and batch (historical) modes with equal reliability and expressiveness. Requires coding. Concepts
  • Cloud Dataflow2: provides a single programming model you can apply to get all your data into a consistent format for your analytics database, whether it is coming in as a continuous stream from your game client or being batch-loaded from your server logs. As incoming analytics event velocity grows, Cloud Dataflow grows as well, dynamically scaling to meet your latency needs. In addition, Dataflow's programming paradigm design brings simplicity and easy of maintenance to your ETL code. Recently open-sourced to the Apache Foundation, you can be confident that your engineering investment in your analytics ETL pipeline can be leveraged on whatever platform fits your business needs - including on-premises or multiple cloud providers.
  • Cloud Dataprep: by Trifacta is an intelligent data service for visually exploring, cleaning, and preparing structured and unstructured data for analysis. Because Cloud Dataprep is serverless and works at any scale, there is no infrastructure to deploy or manage. Limited amount of cleaning technics. Cloud Dataprep by Trifacta enables you explore and process your data without the need for writing code. Google Cloud’s machine learning APIs and services make for easy analysis and data mining.
  • Cloud Dataproc: A faster, easier, more cost-effective way to run Apache Spark and Apache Hadoop. Zone specific
  • cloud Firestore: NoSQL database built for global apps Cloud Firestore is a fast, fully managed, serverless, cloud-native NoSQL document database that simplifies storing, syncing, and querying data for your mobile, web, and IoT apps at global scale. Its client libraries provide live synchronization and offline support, while its security features and integrations with Firebase and Google Cloud Platform (GCP) accelerate building truly serverless apps. Used for Auth/player database in the Gaming Backend Database [https://cloud.google.com/firestore/]
  • Cloud Functions: Event-driven serverless compute platform [https://cloud.google.com/functions/]
  • Cloud IoT Core: Cloud IoT Core is a fully managed service for securely connecting and managing IoT devices. Using the MQTT or HTTP bridge, IoT devices can connect to Google Cloud using per-device public/private key authentication and exchange data. Incoming device data is published to a Cloud Pub/Sub event stream. [https://cloud.google.com/iot-core/]
  • Cloud KMS: is a cloud-hosted key management service that lets you manage cryptographic keys for your cloud services the same way you do on-premises.
  • Cloud Messaging:
  • Cloud Pub/Sub: is a simple, reliable, scalable foundation for stream analytics and event-driven computing systems. You can send and receive messages between independent applications and syndicate data across projects and applications running on cloud, on-premises, or hybrid environments. [https://cloud.google.com/pubsub/]
  • Cloud Run:
  • Cloud Scheduler:
  • Cloud Storage: Cloud Storage is a unified object storage solution that allows worldwide storage and retrieval of any amount of data at any time.
  • Cloud Tasks: Cloud Tasks is a fully managed service that allows you to manage the execution, dispatch, and delivery of a large number of distributed tasks. Using Cloud Tasks, you can perform work asynchronously outside of a user or service-to-service request. https://cloud.google.com/tasks/
  • Compute Engine managed instance groups: A managed instance group (MIG) contains identical instances that are based on an instance template. MIGs maintain high availability of your applications by proactively keeping your instances available, that is, in the RUNNING state. Managed Instance Groups https://cloud.google.com/compute/docs/instance-groups/
  • Compute Engine unmanaged instance groups: are collections of instances that exist in a single zone and do not share a common instance template. You can use unmanaged instance groups to accommodate your pre-existing configurations for load balancing tasks.
  • Container Registry: is a private Docker repository that works with popular continuous delivery systems.
  • Data Catalog: is a fully managed and scalable metadata management service that empowers organizations to quickly discover, manage, and understand all their data in Google Cloud.
  • Datastore: NoSQL document DB. Useful for user profiles, product catalogs, game state
  • Debug:
  • Cloud Deployment Manager: Google Cloud Deployment Manager allows you to specify all the resources needed for your application in a declarative format using yaml. You can also use Python or Jinja2 templates to parameterize the configuration and allow reuse of common deployment paradigms such as a load balanced, auto-scaled instance group. Treat your configuration as code and perform repeatable deployments.
  • Error Reporting:
  • Filestore: Cloud Filestore is a managed file storage service for applications that require a filesystem interface and a shared filesystem for data. Filestore gives users a simple, native experience for standing up managed Network Attached Storage (NAS) with their Google Compute Engine and Kubernetes Engine instances. The ability to fine-tune Filestore’s performance and capacity independently leads to predictably fast performance for your file-based workloads. [https://cloud.google.com/filestore/]
  • Google Data Studio: is a fully managed visual analytics service that can help anyone in your organization unlock insights from data through easy-to-create and interactive dashboards that inspire smarter business decision-making.
  • IAM: users authentication
  • Cloud Endpoints: Develop, deploy, protect, and monitor your APIs with Cloud Endpoints. An NGINX-based proxy and distributed architecture give unparalleled performance and scalability. Using an OpenAPI Specification or one of our API frameworks, Cloud Endpoints gives you the tools you need for every phase of API development and provides insight with Stackdriver Monitoring, Trace, and Logging. [https://cloud.google.com/endpoints/]
  • Cloud Identity: A unified identity, access, app, and endpoint management (IAM/EMM) platform that helps IT and security teams maximize end-user efficiency, protect company data, and transition to a digital workspace. Connects to both AD and Azure AD [https://cloud.google.com/identity/]
  • Kubernetes Engine: is a powerful cluster manager and orchestration system for running your Docker containers. gcloud for cluster level works
  • Stackdriver/Cloud Logging: Stackdriver Logging allows you to store, search, analyze, monitor, and alert on log data and events from Google Cloud Platform and Amazon Web Services (AWS). Our API also allows ingestion of any custom log data from any source. Stackdriver Logging is a fully managed service that performs at scale and can ingest application and system log data from thousands of VMs. Even better, you can analyze all that log data in real time. [https://cloud.google.com/logging/]
  • Memorystore: Fully-managed in-memory data store service for Redis Cloud Memorystore for Redis provides a fully managed in-memory data store service built on scalable, secure, and highly available infrastructure managed by Google. Use Cloud Memorystore to build application caches that provides sub-millisecond data access. Cloud Memorystore is compatible with the Redis protocol, allowing easy migration with zero code changes. [https://cloud.google.com/memorystore/]
  • Stackdriver/Cloud Monitoring: Stackdriver Monitoring provides visibility into the performance, uptime, and overall health of cloud-powered applications. Stackdriver collects metrics, events, and metadata from Google Cloud Platform, Amazon Web Services, hosted uptime probes, application instrumentation, and a variety of common application components including Cassandra, Nginx, Apache Web Server, Elasticsearch, and many others. Stackdriver ingests that data and generates insights via dashboards, charts, and alerts. Stackdriver alerting helps you collaborate by integrating with Slack, PagerDuty, and more. [https://cloud.google.com/monitoring/]
  • Network Load Balancer:
  • Network services:
  • Source Repositories:
  • Cloud Spanner: Cloud Spanner is the first scalable, enterprise-grade, globally-distributed, and strongly consistent database service built for the cloud specifically to combine the benefits of relational database structure with non-relational horizontal scale. This combination delivers high-performance transactions and strong consistency across rows, regions, and continents with an industry-leading 99.999% availability SLA, no planned downtime, and enterprise-grade security. Cloud Spanner revolutionizes database administration and management and makes application development more efficient. Costs a LOT. [https://cloud.google.com/spanner/]
  • Cloud SQL: OLTP workload
  • Stackdriver Profiler: Continuous CPU and heap profiling to improve performance and reduce costs - https://cloud.google.com/profiler/
  • Stackdriver Trace: Stackdriver Trace automatically analyzes all of your application’s traces to generate in-depth latency reports to surface performance degradations
  • VPC service controls: allow users to define a security perimeter around Google Cloud Platform resources such as Cloud Storage buckets, Bigtable instances, and BigQuery datasets to constrain data within a VPC and help mitigate data exfiltration risks.
  • Shared VPC: Shared VPC allows an organization to connect resources from multiple projects to a common VPC network, so that they can communicate with each other securely and efficiently using internal IPs from that network. https://cloud.google.com/vpc/docs/shared-vpc
  • VPC peering Google Cloud Platform (GCP) Virtual Private Cloud (VPC) Network Peering allows private RFC 1918 connectivity across two VPC networks regardless of whether or not they belong to the same project or the same organization. https://cloud.google.com/vpc/docs/vpc-peering
  • Cloud datalab: Cloud Datalab is simple-to-use big data visualization tool that uses the Big Query SQL dialect and simple python imports to put your data into a visualiations quickly and easily. Build a library of Datalab visualizations to use as an up-to-the-minute dashboard, or experiment with new queries in the tool before moving them over to your existing visualization suite. Use these graphs or charts to identify places inducing player churn, or find outliers who may be exploiting broken game mechanics. [https://cloud.google.com/datalab/]
  • Firebase: a comprehensive (mobile) app development platform [https://firebase.google.com/]
  • Cloud Developer Tools: Google Cloud Platform provides a collection of tools and libraries that help you develop quicker. We’re developers too and we love tools that make us more productive, so we’re happy to offer these for use on GCP. Whether you’re looking to manage your resources from the command line, need better ways to debug source code in production, need a solution for running API backends or just want intuitive integration into your favorite IDE, we’re constantly working on tooling to make your life easier. [https://cloud.google.com/products/tools/]
  • Stackdriver Workspace: Workspaces organize monitoring information in Stackdriver Monitoring. Using Workspaces, you can monitor important resources, regardless of where they are. In the Stackdriver Monitoring console, you see the current Workspace at the top of each page: https://cloud.google.com/monitoring/workspaces/

Some other interesting pages and images

comments powered by Disqus