Skip to content

Latest commit

 

History

History
733 lines (576 loc) · 41.4 KB

README.md

File metadata and controls

733 lines (576 loc) · 41.4 KB
Raw

LambdaClass Hacking Learning Path

Table of Contents

You can choose a life of ease and comfort or you can choose a life of service and adventure - Jeff Bezos

My algorithm has always been: You put smart people together, you give them a lot of freedom, create an atmosphere where everyone talks to everyone else. They're not hiding in the corner with their own little thing. They talk to everybody else. And you provide the best infrastructure. The best computers and so on that people can work with and make everyone partners. - Jim Simons

Top-down management leveraging command-and-control hierarchies are for the mahogany boardrooms of yesteryear. We are navigators, adventurers, and explorers of the future. We are married to the sea - Yearn's Blue Pill

A boring codebase doesn't make a bored developer, on the contrary, it frees developers up to think about important stuff and deliver value to the business. Just as I want my language to be boring so I can focus on interesting stuff, I also want my tech stack to be boring - the interesting bits should be in the value added, not the stuff under that.

Introduction

Welcome! This is a guide intended to aid employees in their first days in the company. It will guide you in setting up expectations and working tools, and a whirlwind tour of the background knowledge and tools necessary to work on the kind of projects common at LambdaClass.

The learning path is structured as a series of topics and exercises.

flowchart TB
    start([Start])
    culture([Culture])
    devenv([Development Environment])
    linux([Linux])
    python([Python])
    containers([Containers])
    databases([Databases])
    fp([Functional Programming])
    debugging([Debugging])
    testing([Testing])
    core([Core])
    erlang([Erlang])
    elixir([Elixir])
    phoenix([Phoenix])
    o11y([Observability])
    cloud([Cloud Environments])
    ci([Continuous Integration])
    rust([Rust])
    perfeng([Performance Engineering])
    
    finish([Conclusion])
    advanced([Advanced])

    start --> culture
    culture --> devenv
    devenv --> linux
    debugging --> core
    core --> erlang
    core --> o11y
    core --> rust
    phoenix --> finish
    ci --> finish
    perfeng --> finish
    finish -.-> advanced

    subgraph "Foundations"
      linux --> python
      python --> containers
      containers --> databases
      databases --> fp
      fp --> testing
      testing --> debugging 
    end
    
    subgraph "BEAM Ecosystem"
      erlang --> elixir
      elixir --> phoenix
    end
    
    subgraph "Distributed Systems"
      o11y --> cloud
      cloud --> ci
    end
    
    subgraph "Systems Programming"
      rust --> perfeng
    end
Loading

Our Worldview

Gaming

Machine Learning

Cryptography

Company Culture

At Lambda we won't micromanage you because we don't want to: we give flexibility but ask for responsibility in exchange. Of course, you can have days off to study, or if you feel sick, but you can also have days off if you feel brain fog, nervous, or a bit under the weather; we actually prefer it that way.

If you feel healthy and comfortable, you will be able to solve more efficiently the problems you have in front of you. And even better, you will contribute to a healthier workspace.

Nobody wants to give up, that's a common emotion. The good news is that you are not giving up, you are getting to know yourself better, so you can recharge your energies and keep learning and improving. In turn, when you are working we expect you to hit goals, be proactive, and be a valuable member of the team.

There are three key elements we want you to have present at every moment: Communication, Accountability, and Empathy.

  • Communication is a key aspect of every human relationship. If you can measure your words to give a positive clear message, you can achieve anything in life through teamwork. Try labeling your emotions before communicating, since it is likely that people in front of you do not know what is going through your head. Also, be transparent but assertive if you disagree about something, it will help understand your point of view. Finally, try to avoid direct messages, working on channels allows more people to be in the loop and better feedback can be given.
  • Accountability. Everyone makes mistakes, learners do something about them. If you get something wrong, communicate poorly, or even feel overwhelmed about a situation, raise your hand and say it. It's very likely that a process failed, not you. Many people will be there to help you sort out the issue, and you will surely learn something along the way. If you keep quiet and don't say anything you will regret it later when the truth burst through another hole.
  • About empathy, we are all human beings and have complex emotions, if you are feeling great, it doesn't mean your colleague isn't feeling like shit. Perhaps your coworkers don't feel comfortable talking about emotions, so go talk with them and help them. You can talk to them or their manager if you feel something is going on with them. There is a time when everybody feels weak and needs someone else for support. So be a hero, be empathic.

If you are reading this, you are probably giving your first -or second- steps on the long journey towards being a good developer, but first try to be a good human being and you will see how far that can get you.

Learn to fail

As with most projects making mistakes fast and loudly will make you learn faster, so you should not be ashamed to work directly on a repository through Pull Request rather than workshopping in a draft made elsewhere

  • Avoid google docs. Work directly with Git, it will help get more eyes on your work to get corrections earlier.
  • Default to git. As said earlier, Git snapshots your work and makes it more accessible to the public, more eyes means more people eager to help you

Principles for success and teamwork philosophy

We follow a code of conduct that must be followed to ensure a safe space in the team. You signed it when you entered Lambda and we expect you to follow it. Basically, treat everyone with respect.

Even if you work with a client you're working for Lambda at the end

It's commonplace in Lambda to work collaboratively with clients in the development of apps and features, so maybe you will have more day to day rapport with them than with your other Lambda coworkers. If you're in that situation you should always remember a few key things:

  • Your work relation is with Lambda, not with the client so expect to get feedback from your Lambda's Tech Lead and Project Manager and not from the client, don't ask for it either.
  • The relationship between Lambda and the client is managed by the Tech Leads and Project Managers. If someone from the client side makes a request to you personally via DM please contact your Tech Lead and Project Manager about it. Make sure your Teach Lead and Project Manager are in the loop for all relevant information.

Do The Simplest Thing That Could Possibly Work (aka KISS)

Strive for solving problems in the simplest way possible. To achieve this, you first need to figure out a handful of ways to confront the issue at hand, and only then pick the one you consider will work in the fewest, tiniest steps. Afterwards, refactor. Tomorrow’s code may need to be more complex, so do everything in your power to facilitate tomorrow’s code as simple as possible. Also, while you shouldn't be blind to the future, avoid investing time and effort into developing features that are not currently necessary and might be a waste.

"Always implement things when you actually need them, never when you just foresee that you need them" - Ron Jeffries

Development process/workflow

  • There is a weekly call with every member of the team to set the most important goals of the week, as well as a daily call to focus on the problems of the daily tasks.
  • When working on projects, the specific tasks to tackle are written in tickets on Github Projects. Some actual Lambda's projects are working on other project management software, but these are legacy.
  • The Manager's Path
    • Chapter 1 Estimate reading time: 1 hour

Design and coding standards

  • Suggested reading: Coders at work
  • make is the build tool. Consider these notes.
  • Postgresql is the default database.
  • Write tests.
  • Favor integration tests over unit tests. A project's first tests should be end-to-end smoke and sanity tests, and only after that, if ever, should unit tests be written to test API's, system invariants and help pinpoint bug causes and locations.
  • Do not write tests before you have solved the problem, you may waste time writing tests for the wrong implementation.
  • Write for humans: coding for computers is easy, but writing code that is understandable by another person is an art.
  • Code and document in english unless you have a very specific reason not to.
  • Use meaningful, readable names for variables, functions, and files. Don't try to save characters.
  • Documentation is a sign of the quality of an API. It's easier to write it when the design is right.
  • The less code you have the better. Deleted code is debugged code.
  • Aim for simplicity, not performance. The latter is a by-product of the first.
  • Only introduce optimizations if you have benchmarks that prove an improvement and that the improvement is relevant in the context of the program.
  • Only introduce optimizations if they represent a concrete gain (e.g. cost savings, improved user experience).
  • Follow the Zen of Python, regardless of the language you are using at the moment. English also counts as a language.
  • Don't introduce dependencies prematurely. You must evaluate your requirements, maintenance, and integration costs first.
  • If you want to upgrade a dependency, test it first.
  • Always lock your dependencies. Pin a specific version and a commit of a dependency, don't use the version at master. Use git and commit often, even in one-person projects.

GitHub & GitLab

  • If you are new to GitHub, you can put your hands on with this course
  • Progress in any project must be pushed every day. This must be done within a branch of the master repository and a Pull Request (PR) must be opened for reviewing the code, previous to merging the branch to master.
  • Doc files should always be added via pull request.
    • Be sure those files are written in Markdown.
    • We always use Mermaid for flowcharts, sequence diagrams, graphs, etc.
  • Never push to master directly, and only reviewers can merge branches to master.

Working on open-source projects

  • Suggested reading: The Architecture of Open Source Applications.
  • Use MIT or Apache 2.0 license.
  • Fill the description field at the top of the repo page.
  • Write a decent README.
  • A good readme starts with a succinct description (one or two sentences) and, when possible, a very short and illustrative example use. The rest of the details go after this header.
  • Use continuous integration, most likely GitHub Actions.
  • Make a good balance of features vs maintenance. Maintenance details usually matter more than adding a lot of features.

Tool Stack

We use a suite of tools to facilitate many tasks, as well as to enforce our security standards for all employees.

Please refer to our Notion document, which aims to provide a simple overview of all of our tools.

Community projects

We organize activities where we share thoughts and interests with anyone who wants to join us.

Development Environment Setup

Before beginning with this Journey, if you're a macOS user, you may need some tools or utils for a better experience in your learning path, otherwise, you can skip this section.

  • Two-step authentication is mandatory for every login of the company accounts (e.g. Slack, mail, GitHub).
  • Also, the company's GitHub repositories are accessed via SSH

Homebrew

Homebrew is a package manager for macOS.

GNU tools

Once you have installed Homebrew in your macOS system, you'll need to install some of the GNU tools/utilities for a better work experience.
Just type in your shell the following command lines:

asdf Version manager

  • asdf is a version manager with the idea purpose of generating environmental variables to choose the specific version desired. Remember that to be able to use the environmental variables you need to set their path for the shell to check, you can see how to do it depending on how you installed asdf here.

Code Editors and IDEs

Do's and don'ts about the use of Vertical Whitespace

  • If you'd like to visualize more vertical whitespace than it's established in these Do's and don'ts configure your text editor to show more space.
  • Most of these rules can be enforced automatically in your text editor, configure it to enforce them.
  • Minimize the use of vertical whitespace.
  • Do not end functions with blank lines.
  • Do not start functions with blank lines.
  • Do not use blank lines when you do not have to.
  • Do not put more than one blank line between functions.
  • Blank lines inside a chain of if-else blocks may well help readability.
  • Blank lines at the beginning or end of a function very rarely help readability.
  • Don't leave blank lines at the end of a file.
  • Don't forget to put a single end of the line at the end of a file.

Foundations

Unix Philosophy

Estimate reading time: 2-3 hours

Some questions to guide your learning

  • How does complexity relate to modularity?
  • Why is the text-stream interface important in the Unix Philosophy?
  • Why should design for transparency encourage simple interfaces?
  • How does robustness relate to transparency and simplicity?
  • Even now with video processing, why output of programs should be terse?
  • According to the Unix Philosophy, how noisy do errors have to be?
  • How does the economy of programmer time relate to robustness?
  • Why premature local optimization reduces overall performance?
  • There is the approach of doing things in "one true way", how does it affect extensibility?

Linux

Estimate reading time: 4-5 hours

As you already installed GNU-tools, there's no need to install Linux on a VM (Virtual Machine).

How to try some commands in MacOS with a VM

  1. There’s some commands that doesn’t work in MacOS. If you want to try them, you can use a VM to use Linux. For this, we can use UTM, a Virtual Machine for MacOS. Follow the next steps here.
  2. After the installation is complete, close the VM and press “Stop selected VM”.
  3. Run the VM again.
  4. Now you can use this VM to run the commands.

Some questions to guide your learning

  • What do the following commands do?:
    • ls -l /bin/usr > ls-output.txt 2>&1
    • ls /bin /usr/bin | sort | uniq | less
    • ls /bin /usr/bin | sort | uniq | grep zip
  • How does Linux determine how to interpret the format of a file?
  • What does the sda2 folder represent?
  • What do /root and /usr/bin store?

Networking

Estimate reading time: 5-6 hours

Some questions to guide your learning

  • How are data transmitted over the Internet?
  • What functions do the layers of the OSI model perform?
  • What is the difference between TCP and UDP?
  • What does ARP mean?
  • What range corresponds to private IP addresses?
  • What does IPv6 propose to solve against IPv4?
  • What does IPsec guarantee?
  • What does DNS mean? How does it work?
  • What is the difference between HTTPS and HTTP?
  • What is the difference between asymmetric and symmetric cryptography?

SSH

Estimate reading time: 10 minutes

Before you embark on your Git journey, it is important to learn what an SSH Key (Secure Shell Key) is and how to generate one and add it to your GitHub account. This key will allow you to connect and authenticate to remote servers and services using the SSH protocol. With it, you will be able to connect to GitHub without supplying your username and personal access token each time.

Git

Estimate reading time: 7-8 hours

No one should merge his/her own PR without it being reviewed and approved by a co-worker and/or a client.

Note: commit and push every day. Don't expect something perfect, go for the concrete. In one way or another, you will likely have to iterate later about that work done. Also, since that work isn't only stored on your computer, it won't be lost.

Note on Newlines at end of file It is considered good style - and sometimes a necessity - to always end files with a newline (see here and here). Make sure your editor of choice is correctly configured to add them automatically.

Some questions to guide your learning

  • Why is branching necessary?
  • What is the difference between merge and rebase?
  • What is a stash?
  • What does cherry-pick do?
  • What does reflog do?
  • What does git reset --hard HEAD do?
  • How to get back to a previous commit?
  • How to do a pull request?
  • Why are pull requests important?
  • How to clone a repository using SSH?

Python

Estimate reading time: 3-4 hours

  • HTTP server in Python with Flask
  • Automate the Boring Stuff with Python
    • Chapter 11
  • The Zen of Python is your bible.
  • Use Python 3 in greenfield projects.
  • Strive to migrate to Python 3 in non greenfield projects.
  • Read this to understand environment hell in Python.
  • Use pipenv to escape from environment hell in Python. You can even do this locally in projects that are set up to use virtualenv and virtualenvwrapper.
  • If you also need to stick with a specific minor version of Python (e.g. Python 2.7.14) you can use pyenv in combination with pipenv.

Some questions to guide your learning

  • How do you return how many times a certain character appears in a string using Python?
  • How do you handle different routes for your web app in Flask?
  • Using the logging library, how do you log to a file?

Docker

Estimate reading time: 2-3 hours

Some questions to guide your learning

  • In which scenarios would you use containers and in which you would prefer to use VMs?
  • How do you retrieve and run the latest Ubuntu image?
  • In a Dockerfile, what is the difference between RUN and CMD?
  • Using port 8080, how do you run an image that exposes port 80?

Colima setup for macOS

Install Colima and Docker, then start Colima:

brew install colima
brew install docker
echo "{ \"credStore\" : \"desktop\" }" > ~/.docker/config.json
colima start
$ uname
Darwin

$ colima ssh -- uname
Linux

Sharing Files

$ echo "files under /Users on macOS filesystem are readable from Linux" > some-file

$ colima ssh -- cat some-file
files under /Users on macOS filesystem are readable from Linux

$ colima ssh -- sh -c 'echo "/tmp/colima is writable from both macOS and Linux" > /tmp/colima/another-file'

$ cat /tmp/colima/another-file
/tmp/colima is writable from both macOS and Linux

Databases

SQL and PostgreSQL

Estimate reading time: 9-10 hours

Reference:

Some questions to guide your learning

  • How to use a wildcard as a character?
  • What does COALESCE do?
  • What does LIKE 'S%' do in a query?

Redis

Reference:

Some questions to guide your learning

  • What is the difference between PostgreSQL and Redis?
  • What type of databases are the following? PostgreSQL, Redis, MongoDB, MySQL, HBase, Neo4J, DynamoDB.
  • What makes each database type unique?

Functional Programming

Estimate reading time: 1-2 hours

Referential Transparency

Immutability

Functions as first-class citizens

Debugging

Give me 15 minutes and I'll change your view of Linux tracing

Testing

Unit Testing

Integration Testing

Property-based Testing

Fuzzy Testing

Core

Erlang

Estimate reading time: 4-5 hours

Installing Erlang and Elixir

  • On Mac distributions, you can just brew install erlang to get the latest installation. In case of needing a specific version you can use asdf and install with asdf install erlang [VERSION_DESIRED].
  • Same can be achieved with Elixir through the same means brew install elixir for the latest version, use asdf for a specific version.
  • For other distributions, check here for Erlang and here for Elixir.

Introductory

Language Tutorial

Additional Theory

Some coding guidelines

  • Use rebar3. Include the binary in the repository so it's not an external dependency and the tested version is used. rebar3 is not used directly but through make targets.
  • Support the most recent Erlang version.
  • When building libraries, try to make them both easily usable from the shell and easily configurable via application environment.
  • Indent with two spaces.
  • Avoid using header files (.hrl) [TODO ELABORATE].
  • Supervised processes provide guarantees in their initialization phase, not a best effort. If you expect failure to happen on an external service, do not make its presence a guarantee of your system.
  • Try to avoid timer:sleep on tests, ktn_task:wait_for_success can be a better option. More on this here.
  • Prefer maps to records.

Exercises

Elixir

Estimate reading time: 40 hours

Phoenix

Installing Phoenix

  • For Phoenix, all dependencies and versions needed for installing can be found here. Postgres is recommended as the de facto database. To install older versions of Erlang and Elixir, use asdf to install them.
  • If asdf doesn't let you install a previous version of Erlang this might help.

ExCheck (QuickCheck)

QuickCheck is a testing method (formerly a Haskell library that got adapted to many more languages) that consists of defining expected properties of your program or function, and then testing it extensively against randomly-generated variables automatically. ExCheck is a property-based testing library that resembles QuickCheck.

OpenAPI

The OpenAPI Specification (OAS) defines a standard, programming language-agnostic interface description for HTTP APIs, which allows both humans and computers to discover and understand the capabilities of a service without requiring access to source code, additional documentation, or inspection of network traffic.

Projects

Estimate time: 6-7 hours The idea is to apply everything learned.

Distributed Systems

Observability

Cloud Environments

AWS and other cloud providers

Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

AWS tools

  • IAM: It's used for managing users, groups, access policies & roles.
  • EC2: A web service that provides secure, resizable compute capacity in the cloud.
  • AMI: Amazon Machine Images is "An Amazon Machine Image (AMI) provides the information required to launch an instance".
  • ECS: "Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service. Customers such as Duolingo, Samsung, GE, and Cook Pad use ECS to run their most sensitive and mission critical applications because of its security, reliability, and scalability." In simpler words, it allows you to launch containers on AWS.

(Optional) To further learn about AWS and its practical uses go HERE for exercises. (AWS - EC2/IAM/Containers). These exercises are purely optional and require creating a free tier AWS account, and even then, some of the exercises require a paid account.

Kubernetes

Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available.

Continuous Integration

GitHub Actions

Rust

Estimate reading time: 7-8 hours

Exercises

  • Rustlings. To enable rust-analyzer and its features (such as autocomplete and documentation), run this command in the rustlings directory:
rustlings lsp

Exercise: Conway's Game of Life

Estimate time: 3-5 hours

Implement Conway's Game of Life using Rust + Wasm.

We suggest you:

  • Use the macroquad crate
  • Do not follow the tutorial from section 4 of Introduction to Rust+Wasm (read only 4.3 because the rules are better explained there than on Wikipedia)

and here are some sources you might find useful in this journey:

Exercise: Rusty Merkle Tree

Estimate reading time: 4-8 hours

Implement a simple Merkle Tree in Rust.

  • A Merkle Tree can be built out of an array.
  • A Merkle Tree can generate a proof that it contains an element.
  • A Merkle Tree can verify that a given hash is contained in it.
  • A Merkle Tree can be dynamic, this means that elements can be added once it is built.

We suggest to start simple, do not optimize prematurely because it's pretty sure that you're not gonna need those optimizations.

Here are some sources that you might find useful:

Performance Engineering

Advanced

The articles, book chapters, exercises, and projects here are just the tip of the iceberg, there is a lot more material that unfortunately cannot be covered. Here are some relevant but more advanced topics. There is even more present in the reference material chapter. You can go over all that at your own pace.

Hardware

Estimate reading time: 2 hours

Parsing

Type Systems

Virtual Machines and Runtimes

Algorithmics

Estimate reading time: 15 minutes

Computability

Consensus

Cryptography

Estimate time: 6 hours

  • Introduction To Cryptocurrency A brief introduction to cryptography and P2P networking used in Bitcoin. Avoid The History of Bitcoin module. Also, if you already implemented the Rusty Merkle Tree you can skip the Merkle Trees lecture