No Tests - No Pull Request, Right? Types of Tests that Should Be in Your Codebase

As the blog post Pull Requests: The Good, The Bad and The Ugly claims:

If you do not have time to write tests today - you will find the time for fixing bugs Friday’s night

In other words, to establish solid reliability in production tomorrow we need to invest our time today. Your need for tests for your current project depends on:

  • Size of the team that maintains to the codebase: return True if team.size > 1 else False. Having more engineers means more views on the same items. Tests help to document the opinions how a class or function can be used.
  • Size of the codebase: return True if project.modules > 1 else False. You can’t remember the color of socks that you wore two days ago. Can you remember everything in the project?
  • Duration of development and maintenance phases of the project. The script that you run only once can perfectly live without a solid test coverage. If you’re building a system for decades - please, prepare a good legacy for the next generations of developers.

I have a strong feeling that you think that your code needs tests since you’re still reading this.

In the blog post, I will guide you thru types of automated tests that should be implemented by software engineers: unit, integration, external, and performance ones. It does not cover testing efforts by quality engineers, but the article can still be valuable for them.

You will find code examples that use Python, but you do not have to know the language.

The SQL I Love <3. Efficient pagination of a table with 100M records

I am a huge fan of databases. I even wanted to make my own DBMS when I was in university. Now I work both with RDBMS and NoSQL solutions, and I am very enthusiastic with that. You know, there’s no Golden Hammer, each problem has own solution. Alternatively, a subset of solutions.

In the series of blog posts The SQL I Love <3 I walk you thru some problems solved with SQL which I found particularly interesting. The solutions are tested using a table with more than 100 million records. All the examples use MySQL, but ideas apply to other relational data stores like PostgreSQL, Oracle and SQL Server.

This Chapter is focused on efficient scanning a large table using pagination with offset on the primary key. This is also known as keyset pagination.

Never Give Up, Retry: How Software Should Deal with Failures

It’s doubtful that your backend has everything within one process: you need to read configuration, store customers’ data, write logs and metrics about the status of your software.

If you’re working on a network application - it’s even more complicated: your database can be far far away from the running code.

Some things can go wrong: a network blip might happen, the remote database can be overloaded by incoming requests, a query can reveal some bug in the DBMS and crash it, your data can be out of order on that side because of some reason, and so on.

Microservice architecture encourages cross-process communications over the network. Now your service asks another one for its configuration, that is stored somewhere in the database. You should prepare the software to non-deterministic failures that might occur during the data transfer. And not only then.

In the blog post, we will look into some common failures that can be solved with proper retrying. The basic ideas are described using Python, but experience with the language is not required for understanding.

What Is a Highload Project?

Highload. It was the main buzzword for me 5 or 6 years ago. Since The Social Network movie was released, I wanted to develop such kind of software.

The domain area did not matter for me then: dating services for founders of dating services, illegal online casinos or websites which stream questionable video content - everything would be okay. I wanted to be a part of the team which solves complex engineering problems in scale and delivers product to many thousands and millions of users simultaneously.

I had read dozens of definitions on the Internet from different sources. But I did not understand what does highload mean. And now after years of development of various highload projects I created my very own definition of highload.

Pull Requests: The Good, The Bad and The Ugly

I remember that at my first paid software job we did not have a version control system: all code was uploaded to a folder on FTP server.

Source Code Management was not very sophisticated: we just had the previous revision of the most important source files suffixed with .old in the same directory. Now having ‘just Git’ without a fancy dashboard like the one supplied by Bitbucket, GitHub, or GitLab does not look suitable for me. The tools save a lot of time and are extremely convenient.

Working on our product, a software engineer submits pull requests almost every day. During the last year, I spent approximately 200 hours doing code review - it’s more than 1 month of work!

I believe that merging good pull requests and declining ugly is essential for the success of your product.

What about bad ones? Well, we can do some work to make on them either good or ugly. Let’s review the examples representing different aspects of a pull request. Some ideas are explained using Python, but they are applicable for any other non-esoteric programming language.

🌮 Tacos Delivery Over HTTP/2

Recently I looked into HTTP/2 and its comparison with HTTP/1.1. The adoption of the technology is growing - 16.1% of top Alexa websites already use the latest version of the protocol.

I wanna understand HTTP/2 better. For now, I do not see any project to apply the technology in production. But we all know that another great way to learn something - is to teach somebody else. It happened that y’all are selected as the audience for that :)

In teaching and learning, it’s vital to keep things interesting.

Tacos are definitely not boring. In the blog post, we will try to imagine that we live in the world where web servers deliver tacos instead of HTML-pages. Let’s contemplate pros of serving this delicious Tex-Mex food over HTTP/2 instead of regular HTTP/1.1.

Do not read it if you’re hungry!

Scaling MongoDB by Kristina Chodorow

A single-server database can be only in 2 states: up or down. But Mongo DB Is Web Scale, right? :)

When my wife found the book in our city library, she offered me to read it because I’m interested in learning and using various data stores. And the book is 50ish pages so why not read it in a couple of hours?

Being not an expert in MongoDB at the moment of reading (and only reading does not make me an expert) I found some interesting facts about the sharding of document-oriented solutions.

Check my summary and reading notes below. I rated the book 4 of 5.

How To Choose a Technology For a Commercial Project.
Harmful Advice

Your company starts a new project. You and the team are excited about that. The current one became boring, smells a bit poorly, and the amount of duct tape across the codebase is monstrous. A good moment to start a new adventure.

The business scope of the new software looks defined, the budget is allocated, and you’re eventually asked to pick a technology stack for it: programming language, data storages, frameworks, libraries, development process and so on. You learned all mistakes in a hard way and going to make everything right this time.

Are you ready for the new journey? Excellent, I prepared some harmful advice for you. Check out if you already follow them.

Hey all y'all!

From the beginning of the 2017 year, I was thinking about starting a blog. I had a few options for the main subject:

  • BBQ
  • Burrito
  • Backend

Since I love all these things, it was not the easy decision for me, and it took half of the year to make it.