How to Fix OOM Errors in Spark
A detailed guide on understanding and resolving Out of Memory (OOM) errors in Apache Spark.
Recent Posts
Optimising Spark - Joins, Shuffle, and Skew
Learn the basics of Spark’s query execution, including joins, shuffling, and how to handle data skew effectively.
Query Engines
Discover the evolution of query engines like Hive, Presto, and Spark, and learn how they revolutionize data processing with scalability, speed, and versatili...
Creating AWS API Gateway Private Endpoints
Learn how AWS API Gateway Private Endpoints use AWS PrivateLink to securely expose APIs within a VPC, ensuring data stays off the public internet.
Poetry for Dependency Management
Explore how Poetry manages Python dependencies and ensures reproducible setups across systems, enhancing project collaboration
I Built a SaaS Business for a Year. Here’s What I Learnt
Explore the journey of building AI-driven SaaS products for e-commerce, offering essential insights and strategies for aspiring tech entrepreneurs in the dyn...
Auto-GPT is overhyped.
Auto-GPT utilizes OpenAI’s API to perform tasks autonomously but can be overhyped due to its repetitive nature and high cost.
Large Language Models - A Primer
A informative and easy to understand summary on large language models, particularly ChatGPT
Creating an endpoint on AWS Sagemaker with Pulumi
Describing how to provision an endpoint on AWS Sagemaker with Pulumi and Python
Introduction to Infrastructure as Code (IaC)
Explaning the concept of Infrastructure as Code (IaC) and popular IaC tools
Ingesting data from Amazon Redshift to Sagemaker
Ingesting data from Redshift to Sagemaker by configuring IAM and VPC, as well as establishing a connection on Sagemaker
Explain IAM like I’m 5 (AWS)
Understanding basic concepts of AWS IAM such as user, policy, roles.
Why I stopped using Grid Search CV
Tuning hyperparameters in a supervised machine learning model efficiently with Optuna.
Supervised Learning Models for Structured Data
Summary of all the popular supervised machine learning algorithms to classify structured data in tables.
A Framework for EDA
Guide on performing Exploratory Data Analysis (EDA) when you start your machine learning project
ssh into an AWS EMR cluster from scratch
How to create a Virtual Private Cloud (VPC), configure network settings, and subsequently connect to an AWS EMR cluster via ssh.
How to land a data science job without a relevant degree?
A step-by-step guide on getting your first job as a data scientist based on my personal experience.
Why every data scientist should write a blog, and how.
Blogging enables you to share your knowledge and of course, dank memes.