Ephemeral Hadoop clusters using Google Compute Platform

Companion to Learning Hadoop and Learning Spark courses on Linked In Learning

A Python framework for data processing on GCP.

gomrjob - a Go Framework for Hadoop Map Reduce Jobs

Google Cloud Dataproc is a managed Apache Spark and Apache Hadoop service that lets you take advantage of open source data tools for batch processing, querying, streaming, and machine learning.

A search engine to query social media insights with political theme

opens a chrome browser to a dataproc cluster

Creating an Inverted Index of words occurring in a large set of documents extracted from web pages using Hadoop MapReduce and Google Dataproc

Demonstration of Google Cloud Dataproc Workflow Templates

Covers big data processing in cloud

Demonstration of Google Cloud Dataproc for running PySpark jobs

gke with terraform, dataproc with terraform

Twilytics provides insights into the trends on Twitter based on real time data

Demonstration of Google Cloud Dataproc for running Spark jobs with Java

Collection of personal resources on Google Cloud

Collected data about from three sources, one opinion-based social media in twitter, research data in New York Times, and the third is the common crawl data for the same topic or key phrase, and from similar time periods. Processed the three data sets collected individually using classical big data methods like Map Reduce in Google Dataproc Clusters. And then compared the outcomes using popular visualization methods in tableau.

Google DataProc Spark Scala Job for MNIST Handwritten Digit Recognition using Decision Trees (Spark MLlib)

Google Cloud (GCP)

Neo4j, Cassandra, Hadoop, PySpark, RDD, MapReduce, Cluster-Computing, DataProc

Using PySpark for Tensorflow model inferencing on GCP Dataproc Cluster. Demo for PyCon Hong Kong Fall 2020 Presentation

Spark-приложение для потоковой обработки датасета олимпийских достижений и рекордов

Working examples for some components on GCP, and instructions on how to run them.

Under construction....

Dataproc Customisable HA cluster debian-9 with zookeeper,kafka ,BigQuery and other tools/jobs with Terraform

Running a wordcount job on a Google Dataproc cluster

Data Workflows with GCP Dataproc, Apache Airflow and Apache Spark