Shahinism

The Default Playbook Legacy migration has an almost universal playbook: Step 1 - Asset Discovery: Make an export of all assets in the environment to see what is there to migrate.

I found this essay by Nicola Ballotta quite interesting.

I favored Nushell.

Introduction One of the challenges of having a distributed solution for data platforms is testing the system end to end.

Introduction Nowadays, Great Expectations is a very viable option for most of the organizations to introduce data quality solution for their data platform.

Transitioning from a web development background to data engineering, I’ve encountered a significant cultural shift.

It all started from a change in the checkpoint path of our Spark applications.

In PySpark’s structured streaming, striving for modular, reusable code while ensuring adaptability for each unique use case is a common challenge.

Lately, our team has embarked on a journey to enhance our data storage infrastructure.

Over the past year, I’ve dived deep into the world of NixOS, discovering its robustness, flexibility, and power.

Deep within each of us, there’s a burning desire—a fundamental drive that shapes our actions and decisions.

Today, I discovered that when utilizing kafka-console-consumer and including the --group flag without specifying --from-beginning, the offset is set to the latest message.

Over a decade ago, big data became a buzzword, and everyone envisioned a future where data-driven decision-making would be the norm.

When submitting a commit command, I was facing: error: gpg failed to sign the data fatal: failed to write commit object After some searching, I managed to enable Git’s debugging log as: GIT_TRACE=1 git commit And the output changed to: 08:55:55.059810 g…

Overview In this post, we are going to review how to run a Spark application, as a single node Fargate task.

Overview In this article, we will talk about the second-ugliest exception in the history of programming and attempt to handle it in our Spark apps.

Overview Virtual Private Clouds (or VPC) as you all probably know, is one of those services, which would be a lifesaver when you know how to use them.

Whenever working on a new source code, I find it quite cumbersome to grow to enough understanding around it to be productive with it both in terms of security and speed of delivery.

TLDR: To practice my Emacs-Lisp-fu and revise my terminal workflow, I implemented a Guake-like pop-up terminal inside Emacs.

I never was a fan of scripting, and unless it was a real edge case, I was always ending up refactoring my needs based on existing tools instead of scripting.

Preface It’s been a while I’m using aws-vault to manage my AWS credentials for accessing the different client accounts.

Challenge In this post I’m going to talk about the permutation challenge.

Challenge The challenge I tend to review in this post is called hash map.

Updates [2019-12-23 Mon]: Added Rust Solution. [2019-12-21 Sat]: Added Clojure Solution.

Challenge The first challenge we pick to solve is Compress Challange.

Overview In this series of blog posts, I plan to solve programming challenges I find around the internet, in books or throughout my daily job in different languages.

How to not use Google search Prelude It’s almost about 6 months since the last time I used Google search as my main search engine.

Run Python test suites without leaving Emacs Recently, I’m trying to overcome my laziness and incorporate more automated testing into my software development procedure.

My Journey to Privacy A decade ago, it was easier for ordinary people to start using the internet without having any concerns about being spied on.

First Post A couple of years ago I was considering myself a blogger, with a decent archive of the blog posts (all in Persian) on topics that were interesting to me at the time.

I was watching a lecture on computer security recently and came up to this website called shodan.io.