Skip to content

Category: Algorithms and Programming

Part 2: Product Data Models – Ethereum Payment

In this series of articles, I’m doing a walkthrough of a hobby project I’ve worked on, a payment system for the Ethereum blockchain. Go read the intro article for more context about this project.

In this part, I will cover the following topics:

  • Product details, whitelistings, coupon codes
  • Displaying purchases and subscriptions, products available, and past purchases
  • Data models for products and whitelistings

None of what is described in this article is about web3, it’s all really web2 development. However, I do have to describe it to lay the foundation for parts 3 and 4 that will be focused on interacting with the blockchain, and with a recap of what I’ve learned about what crypto wallets and blockchain for developing mass-consumer product experiences.

Below are the data models used for products, purchases, and whitelistings and coupons (click to enlarge).

The data schemas used to represent products, purchases, and whitelists/coupons (click to zoom)

The follow diagram shows three user flows: the first flow shows products for non-signed in users, the second flow for signed in users, and a third flow just to display the current status of the access rights.

System diagram showing three user flows to display available products, past purchases, and current access rights.

Part 1: Authentication and Crypto Wallet Management – Ethereum Payment

In this series of articles, I’m doing a walkthrough of a hobby project I’ve worked on, a payment system for the Ethereum blockchain. Go read the intro article for more context about this project.

In this part, I will cover the following topics:

  • Connecting a crypto wallet via a frontend, and detecting states in MetaMask
  • Backend endpoints needed to manage wallet connections, wallet signature, and JSON Web Tokens.
  • Data models for users.

You will find that there isn’t much complexity in this article. Authenticating a crypto wallet is most about calling methods from the wallet API from the frontend, and then passing data around between the frontend and the backend for verification.

Connecting a Crypto Wallet

Whatever software you build that will have to interact with the Ethereum blockchain, either reading data or transacting, you’ll need to connect to an Ethereum node to access the Ethereum blockchain.

For most use cases, you won’t be running your own Ethereum node, and you won’t be calling directly any Ethereum node either. Instead you’ll use a third party that offers a centralized point of access to the blockchain: you connect to their backend servers, which are connected to the blockchain. It’s kind of counter-intuitive that we have to use centralized third party providers when you think about it given that one of the core tenets of the blockchain is that it’s fully decentralized, but that’s what we have to deal with these days.

As a human user of the blockchain, you need a crypto wallet to initiate various actions and execute smart contracts. Every Ethereum client implements the same JSON-RPC specifications, which is the standard that applications can rely on to interact with the blockchain.

Building a Payment System for the Ethereum Blockchain

Over the past few years, I’ve oriented the content of this blog towards engineering management and tech leadership, and consequently, I haven’t written anything purely technical for a while. Some of my most technical article series ended up finding a large audience, namely Implementing a Key-Value Store, and Coding for SSDs, and it now feels like it’s been an eternity since I wrote them.

Crypto protocols have made a serious dent in the tech space, and even more so over the past couple of years, often referred to as a whole as crypto or web3 technologies. With my personal commitment to keep upskilling myself, I have decided to make it a top priority for me to understand how blockchain technology was built and operated, and what kinds of businesses and product experiences this technology could enable in the long term.

I decided to build something basic yet complex enough that it would force me to interact with blockchain data in interesting ways. Because token transactions and crypto payments are simplest interactions on the blockchain, I settled on building a payment system.

Therefore in this series of articles, I will be sharing how I implemented a fully-functional payment system on top of the Ethereum blockchain, with the following features:

  1. Managing a list of products and time-bound subscriptions. Allowing for coupons and whitelistings.
  2. Displaying the current and past purchased products for a user.
  3. Guiding users through payment transactions on the Ethereum blockchain, by connecting their Ethereum wallets.
  4. Handling complex situations of network and service disruptions, to prevent double payments, avoid hanged transactions, and cover all other edge cases. This is mostly done by making all operations idempotent and idempotency keys.
  5. Monitoring the Ethereum blockchain to verify that payments for a particular product has been received.
  6. Once payment has been received, allowing paid users to access the private parts of a website by signing on with their Ethereum wallet to authenticate.

Six Patterns to Optimize Your Monolithic Codebase

Migrating to microservices isn’t always needed: first, you must fix your monolith.

The tech industry, and software engineers, are wired for the new and the flashy. That’s also why we see dozens of new javascript frameworks getting created every year, the majority of which end up burning into oblivion as they re-enter the Earth’s atmosphere—well, except for jQuery, which somehow keeps surviving.

The same can be said of architecture: almost every time I hear or read someone mention monolithic architecture, there’s almost always a negative connotation associated with it, because it has this feeling of old and outdated.

But simply because monolithic architectures are older doesn’t mean they’re bad, or that they’re less, compared to service-oriented architecture or to serverless.

This reputation comes from the fact that most monolithic architectures end up turning into giant spaghetti monsters of code, difficult to build, deploy, and evolve. I agree with that, that has also been my experience.

What I disagree with is the speed with which many engineers are ready to discard monolithic architectures entirely and switch to something else, say microservices, which they understand even less, instead of trying to fix what’s already there. There’s a lot of hype involved with such pushes, because it entails using new cool frameworks, new programming languages, and so on.

In addition to that, there’s often a lack of understanding from engineers that the decision of choosing an architecture must never depend only on technical considerations. In fact, it should be the other way around: picking an architecture should always be done by putting most weight on what will best serve the business needs and growth of the company over time, and take technical considerations only as a second-order concern.

And before discarding an entire architecture or codebase, one should always strive to fix it with what’s already there.

In this article, I want to share techniques I’ve seen deployed in production and which have helped make monolithic codebases easier to work with. I’ve seen first-hand that they were successful in improving build times and consequently deployment times.

The six patterns for optimizing your monolithic architecture are:

  1. Remove build bottlenecks
  2. Extract frequently updated code areas into their own modules
  3. Clean up unneeded dependencies
  4. Clean up unused code
  5. Enable server-side caching in your CI/CD
  6. Use subviews to mock dependencies

In the rest of this article, I will share more details about those techniques, and how to go about using them

I’ll be using the term “microservices,” as if I’m taking the perspective of a backend system. But this article also applies to frontend applications and to microfrontend architectures. Also, I’m going to stay very generic in my statements, and that’s intentional: I want to focus on the big picture ideas without being specific to a particular language or toolchain.

A Complete Overview of Front-End Development in 2021

Wasm, ESLink, Webpack, Serverless. Does this ring a bell? Do you know what these technologies are used for?

There are so many concepts and frameworks used by front-end developers, it’s hard to keep track when you don’t work on it every day.

I recently spent time brushing up my front-end knowledge and skills, and I wanted to share what I learned.

In this article, I give a concise yet complete rundown of all the main technologies used for front-end development, along with resources to dive deeper where needed.

I Invested in Myself: I Hired a Copy Editor to Improve My Writing

I invested in myself.

I hired a copy editor to review a 4,000-word draft I was working on. 48 hours and $180 later, he had left 439 edits and comments on my draft, along with a gold mine of feedback on how to improve my writing skills.

Why care about writing? In these times of continued lockdowns and remote work, more communication happens in written form, whether it’s emails, messaging, or long-format reports and articles. More than ever before, your mastery of the written word can boost your career in unexpected ways.

Besides, writing skills will be useful to you in any job you’ll have in the future, and regardless of industries. It’s one of those skills that’s entirely transferable, like public speaking and negotiating.

Hiring an editor to review your writing and giving you feedback is one of the best gifts to offer yourself.

Here I share my experience doing it along with tips on how to make the process as fruitful as possible, hoping it will help you too.

How to Keep Your Tech Skills Sharp in a Leadership Role

When I became a senior engineering manager three years ago and had multiple teams reporting to me, I was no longer building things on the job myself. This was the first day of the decay of my pure technical skills, and with it came the question of what I was going to do about it.

Fast-forward to March 2020, I’m sprinting through the Sao Paulo airport, hugging my carry-on luggage close to my chest and dodging other travelers as best I can. I was visiting South America when COVID-19 hit Europe and air traffic started shutting down. I was heading back home to Amsterdam and my connecting flight from Buenos Aires had landed an hour late. So I made a run for it. If I didn’t catch this flight, I was going to be stranded 10,000 kilometers away from home.

Fifteen minutes, 40 gates, and a wobbly knee later, I finally reached the boarding area completely out of breath and managed to catch my flight home.

When I got back to work in the following days, most of my colleagues had been working from home via video calls for about a week already. I had missed the early days of the quarantine, but luckily the local supermarkets hadn’t been raided too badly, and I was able to get my hands on toilet paper.

As I started setting up my quarantine routine, I made the decision to tackle the hitch that had been annoying me for a long time, and by this I mean I was determined to catch up with the cutting edge in tech.

My plan was simple: I was going to build a web app as a pretext to learn an entire tech stack end to end. I actually tracked the time I spent on it weekly to hold myself accountable, and it worked!

After more than 100 hours of coding and learning between April and November 2020, I launched the MVP for Sidenote.me, a web app to take time-stamped notes on videos. In the process I learned in-depth about TypeScript, Node.js, and MongoDB, and I performed a high-level refresher on the state of the industry in other tech ecosystems, such as containerized infrastructure, micro frontends, and serverless computing.

Launching Sidenote 🚀

If there’s one lesson that 2020 has taught me, it’s the value of focusing on opportunities rather than lamenting myself on what is not possible or what was taken away. I took advantage of the quarantine to go back to coding on a regular basis during my free time, and in the process I learned about TypeScript, Node.js, and MongoDB.

When I took a more managerial path early 2016, my focus naturally switched towards organizational topics. I must admit it did feel that my tech chops were slowly getting rusty, and I knew I had to do something about it. So when the first quarantine started, I set a simple goal for myself: to code for at least three hours per week. I actually tracked the time I spent on it to hold myself accountable, and I did hold onto it.

Today, after more than one hundred hours of coding and learning spent between April and November 2020, I am super proud to announce that I am launching an MVP for Sidenote.me, a web app to take timestamped notes on videos. If you’re reading this post until here, it would mean the world to me if you gave it a try just for a minute and then posted your feedback in the comments section below. And if you like it, talk about it on your favorite social media.

By the way, I’m going to be posting on this tech blog again very soon, about engineering and leadership topics. Subscribe to my newsletter below for fresh articles.

Join my email list

Service Ownership Checklist

Building and maintaining infrastructure services requires to strive for quality and ownership. But it’s not always easy to know what we are missing, and what assumption we are making that we don’t know of. To help myself and my colleagues reason about whether we are addressing the important topics, I came up with something I call the Service Ownership Checklist. It’s still in a draft format, but I’ve already refined it thanks to the help and feedback of many of my peers around me, and I’m now releasing it on my blog hoping that it can help other infrastructure engineers as well.

The way to use this document is to share it with your colleagues and teams, and have them ask each other some of the questions to see how they’re doing on all these topics and challenge their assumptions. You will hopefully uncover unknown issues, and create enough urgency to go and fix them.

This blog post is organized in two parts. The first part is the Service Ownership Checklist, a set of loose questions that can be used in brainstorming and sharing sessions, and the second part is a condensed version in the form of a questionnaire, the Service Ownership Questionnaire, which shows the different levels of quality for each reliability topic.

The SRE organization at Google is running Launch Reviews when they release new services, and for this they use a Launch Review Checklist. I recommend that you read Chapter 27 of the Google SRE book, which covers the subject.

Finally, if you think of any other topics, or of a better way to group the questions into categories, please post a comment below! And if you enjoyed this article, subscribe to the mailing list at the top of this page, and you will receive an update every time a new article is posted.