Running in Production

Meteor Cloud Is a Full Service Hosting Solution for Meteor Apps

Mon, 20 Dec 2021 00:00:00 -0500

In this episode of Running in Production, Filipe Névola goes over building a hosting platform for Meteor apps. It’s hosted on AWS with ECS and has been running in production since 2015.

Filipe talks about building critical services with Go, using Meteor to build front-end web dashboards, the importance of monitoring, using Recurly for subscription payments, multi-region AWS hosting and overall providing a highly available platform for thousands of clients.

Topics Include

1:45 – High level overview of the hosting platform
7:01 – Splitting up compute resources for enterprise clients
8:44 – Motivation for choosing Go for a few of the back-end services
11:24 – Feeding data to the web dashboard with MongoDB, ECS, a load balancer and Go
15:53 – The Go proxy service was built using the standard library (no 3rd party libraries)
17:46 – Differences between the free and paid plans
22:49 – Displaying a custom page if your Meteor app happens to be down
26:28 – Going over a few services, starting with the Docker image builder
31:18 – The Go services are in a mono repo but they can be individually deployed
33:36 – The next service is the scheduler which is custom built
41:49 – How the web dashboard gets updated from events on the back-end
47:08 – The last service we’ll cover is for registering SSL certificates with Let’s Encrypt
50:21 – Monitoring is very important and they’re using Datadog, plus being on-call
54:26 – Postmark and SendGrid are both used to send emails
56:23 – Payments are handled through Recurly
1:00:28 – Loggly is used for logging and a bit of analytics
1:04:08 – Handling a lot of incoming notifications and making sense out of alerts
1:08:05 – Choosing AWS for hosting everything and using ECS over EKS
1:11:20 – It’s hosted across multiple AWS regions (Virginia, Ireland and Sydney)
1:15:55 – The open source side of Meteor is very very important
1:17:49 – How Terraform is being used to manage their infrastructure
1:20:31 – ScaleGrid is used to host their MongoDB instances
1:22:29 – How clients store their secrets
1:24:15 – How deployments are handled from development to production
1:32:34 – All data is backed up on a regular basis with lots of redundancy
1:36:01 – Handling big traffic spikes with little warning
1:38:38 – Best tips? Monitor everything and avoid premature optimization
1:41:43 – Check out Filipe on Twitter, his coding education platform and YouTube channel

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

CompanyCam Helps Contractors Document Their Job

Mon, 22 Nov 2021 00:00:00 -0500

In this episode of Running in Production, Chad Wilken goes over building a service to help contractors document their job and communicate with their crew. It’s been up since 2014.

Chad talks about handling ~800k photo uploads per day, building a Rails API driven app, creating a great mobile experience with React Native, handling millions of daily Sidekiq (pro) jobs, transitioning to AWS ECS with Fargate, deploying with Capistrano and so much more.

Topics Include

3:04 – Motivation for switching from .NET to Rails
5:59 – It’s a Rails API with a React front-end, first class mobile support is important
9:34 – What type of screens does this app have?
11:58 – As the CTO, Chad doesn’t get a chance to code as much as he used to
13:15 – A few Rails features and gems being used (interactor, wisper and more)
16:35 – Third party sites that they integrate with
18:11 – Contracting companies are a lot more technical than you might think
22:39 – Every photo upload hits the Rails back-end but it’s not processed there
25:30 – Millions of jobs are being run through Sidekiq per day and payments
28:46 – Spending time on the company admin back-end vs user features
33:20 – Postgres (RDS), MongoDB, Elasticsearch and Redis are being used
36:13 – The Flipper gem is being used for feature flags
37:44 – Docker is being used in development with a transition to using them in production
39:30 – How things are currently deployed and where they’re going with AWS Fargate
41:03 – From Chef to Ansible with Capistrano handling app deployments
43:35 – Ubuntu is their distro of choice
44:00 – Moving over to ECS with Fargate and hiring dedicated ops folks
48:28 – A run down of which AWS resources they’re using
49:25 – The deployment process from development to production
54:59 – All developers on the team can use whichever OS they prefer
55:37 – Handling backups of all user data
58:22 – Logging, error monitoring and alerts
1:01:40 – Best tips? Design patterns are important, so is caching and saying no
1:03:48 – Their AWS bill is about $38,000 a month
1:05:03 – Check out https://companycam.com, they’re hiring!

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

TinyPilotKVM Lets You Remote Control Your Server from Your Browser

Mon, 18 Oct 2021 00:00:00 -0400

In this episode of Running in Production, Michael Lynch goes over building a hardware device that lets you remote control your server without installing any software. It’s been available since mid-2020.

Michael talks about how it works, using Ansible to provision a Raspberry Pi, Using Flask with SocketIO, rendering 30 frames per second at 1080p with under 200ms latency, using web components, selling his devices on Shopify, hiring quality freelance developers and more.

Topics Include

7:13 – The process to build a custom piece of hardware
12:33 – 3D printing a custom case
15:16 – The assembly process and selling about 150-200ish devices a month
16:20 – Ansible and a Bash script ensure everything gets installed on the device
20:16 – Everything including the OS can run on about 1.5GB of memory
22:01 – Motivation for using Flask and Python
25:29 – Using Python’s built in testing library and a few useful 3rd party packages
26:47 – Web components are being used quite heavily
31:44 – Getting started with web components
34:34 – Rendering everything at 30 frames per second at 1080p with SocketIO
38:34 – The differences between the regular and pro version
40:57 – There’s no bundling for JS and CSS
43:11 – Docker was considered but wasn’t included in the end
46:44 – Purchasing the pro version and ancillary services around the hardware
50:52 – Shopify is used as a store front and they make about 20-30k a month
54:26 – The deploy process and reviewing code from their freelance devs
1:00:09 – Hiring quality developers by making it a great place to work for
1:02:05 – Getting new versions of the code on user’s devices
1:07:35 – Is it possible to brick your own device? It hasn’t happened yet
1:11:40 – Best tips? Incrementally build and release your product as you go
1:16:26 – How Michael hired a few initial freelance developers
1:21:41 – Check out TinyPilotKVM, Michael has a blog and is on Twitter too

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

uscreen Is a Platform That Helps Content Creators Build a Business

Mon, 11 Oct 2021 00:00:00 -0400

In this episode of Running in Production, Nick Savrov goes over building a platform to help content creators build a business with Ruby on Rails. It’s hosted on Heroku and has been up and running in production since 2014.

Nick talks about supporting 6.5 million users, using Turbolinks, having a 19 developer team working on a monolithic app, sending millions of weekly emails, storing billions of weekly events, using ShapeUp to help manage the project and much more.

Topics Include

2:49 – They support 6.5 million users on the platform which helped creators make $100m+
4:19 – There’s 2 parts to the system, the admin for creators and the user experience
8:49 – BREAKING NEWS: Rails can scale and it’s working out nicely for them as a monolith
10:47 – Turbolinks and Liquid helped their app a lot
14:10 – It feels like a Rails app but uses VueJS and InertiaJS on the front-end
18:47 – A couple of interesting gems in their Gemfile, including using Fastly
21:38 – They have a great relationship with Mux for streaming video
25:25 – Approaching the latest stable version of Rails
27:05 – 19 Rails engineers are working on the monolithic code base
28:33 – Payments are supported with Stripe, PayPal and Authorize.net
31:42 – About 10 million emails are sent out every week through SendGrid
34:29 – Custom domains for customers and using Heroku with nginx / Let’s Encrypt
39:51 – Postgres is the main database along with ClickHouse (billions of events a week)
42:33 – What types of events are being logged and how can it be viewed?
46:20 – A custom nginx router that sits in front of Heroku
50:11 – Reasons for using Heroku and setting up an auto-scaler
54:51 – A couple of Heroku add-ons and using New Relic (NOTE: 10 billion rows = 250GB)
57:38 – The deployment process from dev to prod (CI, code reviews, GitHub discussions)
1:07:57 – What is ShapeUp which is Basecamp’s methodology around project management
1:09:06 – Backing up user data and more info about ClickHouse
1:14:51 – There’s value in performing soft deletes as long as you really delete it later
1:16:03 – OpsGenie, New Relic and Uptime Robot are used for alerting and being on-call
1:22:34 – Best tips? Follow standards when you can such as using built in Rails features
1:26:03 – You can find Nick on GitHub and Telegram

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Great Question Makes It Easy for Teams to Perform Customer Research

Mon, 04 Oct 2021 00:00:00 -0400

In this episode of Running in Production, PJ Murray goes over building a customer research app with Ruby on Rails. It’s hosted on Heroku and has been up and running since mid-2020.

PJ talks about using feature flags, integrating Stripe with Jumpstart Pro, building out a React front-end, the value of having business metrics, taking data security very seriously, having a pragmatic approach around test coverage and tons more.

Topics Include

4:16 – It took about a month to get the MVP out and a few months after for it to be sellable
6:52 – Motivation for using Rails
8:54 – How the app works and going over some of its screens
13:21 – ActiveJob is such a good abstraction it’s easy to forget what job library you use
14:14 – A bunch of useful gems that are being used
18:14 – The user experience can often impact the technical complexity of what you’re building
21:53 – Feature flags and swinging back to JSONB columns
26:21 – Stripe handles all of the payments
28:24 – The app is pretty much one big monolith and it’s a good thing
29:55 – About 25k lines of Ruby code and 40k lines of Typescript on the front-end
33:04 – If Turbo were around for years would you have used it over React?
36:37 – You shouldn’t be afraid to touch code in your codebase
38:42 – Getting a decent amount of things planned out before implementing the code
44:00 – Postgres, Redis, Mixpanel and Datadog for app and business alerts / logging
49:19 – Limiting access to production data from developers
51:17 – Heroku helped them get to market faster and they had YCombinator credits
54:36 – The deploy process from development to production
1:01:20 – Limiting access at the GitHub repo level and Heroku
1:04:48 – In general backups are handled by the providers they use (Heroku, S3, etc.)
1:07:21 – Heroku will send out alerts if something unexpected is happening with the site
1:09:31 – Best tips? Be pragmatic about testing and code coverage
1:11:58 – User design and UX is handled by a specific team member
1:14:09 – Check out https://greatquestion.co and they’re hiring too

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Robot Accounts AI Provides a System to Let You Categorize Invoices

Mon, 27 Sep 2021 00:00:00 -0400

In this episode of Running in Production, Josh Kinabrew goes over building an AI driven invoice categorization system using Ruby on Rails. It’s hosted on Heroku and AWS and has been up and running since 2013.

Josh talks about training an AI system to scan and break down pictures of invoices, managing thousands of clients, using the latest stable version of Rails, using Sidekiq Pro, interfacing with QuickBooks and more.

Topics Include

1:52 – A little bit about how the service works
5:32 – Training an AI to process invoice items
8:33 – Thousands of clients and the motivation to use Ruby on Rails
10:28 – Using AWS Rekognition’s Textract and SageMaker for AI and ML decisions
14:55 – It takes about 2 minutes to get an initial result after uploading an invoice
18:12 – The mobile version is served through a responsive web application
21:36 – They’re using the latest stable Rails release with good test coverage
25:31 – Going over a few types of Sidekiq Pro driven background jobs that are running
29:28 – Action Mailbox was a huge win for them, it’s also a monolithic code base
31:54 – They’re using Heroku’s CI service
32:52 – Good old Sprockets is used for the asset pipeline at the moment
35:31 – Josh really likes Postgres, Redis is also being used quite a bit
40:22 – What it’s like to work with QuickBook’s imports and exports
46:30 – Most things are hosted on Heroku (2 worker and web Dynos)
49:05 – Papertrail, Sentry and Mailgun are being used
51:42 – Before sending emails most agriculture folks were faxing paperwork
52:50 – The deploy process from development to production
54:58 – Terraform is used to spin up the infrastructure, even on Heroku
59:48 – Avoiding real customer data when copying data from prod to dev
1:05:42 – Currently there’s no external monitoring checking the site for up-time
1:07:49 – Best tips? Don’t worry about being perfect, get something out there
1:09:43 – They’re hiring, you can email Josh and they’re on Twitter too

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Fundraze Is a Flexible Fund Raising and Campaign Management Service

Mon, 20 Sep 2021 00:00:00 -0400

In this episode of Running in Production, CJ Avilla goes over building a fund raising platform with Ruby on Rails. It’s hosted on Heroku and has been running in production since 2015.

CJ talks about rewriting an app with Rails, processing $30 million dollars of donations, using Stripe, maintaining a Rails 4.2 app, carefully sending out bulk emails, ensuring good tests are written, keeping things simple and more.

Topics Include

2:26 – Rebuilding a similar app with a new tech stack is easier the 2nd time around
6:14 – Motivation for using Rails and live coding features right next to his client
9:33 – What the app does at a high level and what some of the screens do
12:20 – The app is running Rails 4.2 and it’s humming along with minimal maintenance
13:29 – Stripe handles accepting donations and keeping things as simple as possible
17:33 – Processing over $30 million dollars since the site went live
22:35 – It’s worth checking out which payment providers are available in different countries
24:51 – Using an older version of Stripe’s API but it’s super stable
27:22 – A couple of gems from 2015 which helped build this app
28:50 – (1) standard Heroku Dyno for the web app and (1) 2x size Dyno for the worker
32:15 – Postgres, Redis and Resque are being used along with Heroku’s cron scheduler
35:48 – Sending bulk emails out can be scary
39:42 – Using Rbenv locally to manage things in development
42:42 – The reasons for picking Heroku were mainly to avoid any type of ops work
46:03 – What it’s like to develop a new feature and push it up to production
47:58 – Not doing traditional TDD but tests are still written
51:18 – Heroku is in charge of performing daily database backups
55:40 – Handling background worker spikes with popular donation pages
57:42 – Tagging the current user in Rollbar errors and then emailing customers ASAP
1:00:27 – Best tips? Keeping it simple really drives down your maintenance
1:02:38 – CJ’s has a YouTube channel and a personal site at https://cjav.dev

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Running in Production Is a Podcast Where Devs Chat about Tech Stacks

Mon, 13 Sep 2021 00:00:00 -0400

In this episode of Running in Production, Nick Janetakis goes over building a podcast site with Jekyll and Ruby. It’s hosted on a single DigitalOcean server and has been running in production since October 2019.

Nick talks about what it takes to release an episode, keeping things simple, developing a custom audio player, hosting a bunch of sites on a single DigitalOcean server with nginx, using shell scripts to help reduce human errors and more.

Topics Include

2:06 – The podcast is not sponsored and it’s done in Nick’s spare time after hours
2:30 – What’s involved end to end to put together an episode
8:50 – Each episode gets about 400 downloads or listens but it’s hard to track
10:41 – Motivation for using Jekyll and Ruby
13:03 – A couple of custom Jekyll plugins to help building a podcast site
16:55 – So many static generators to choose from, just pick one
18:09 – Use the tools that you like and don’t constantly second guess yourself
19:34 – What is Liquid (Jekyll’s templating language)?
21:01 – The custom audio player is the only real amount of JavaScript on the site
25:42 – Jekyll-Assets is being used to MD5 tag static file names for cache busting
26:34 – nginx is serving the site with Let’s Encrypt handling the SSL certificates
28:12 – The only SAAS tool being used is Google Analytics but I don’t use it for much
29:20 – A few static sites are all hosted on a $5 / month DigitalOcean server
32:06 – The server load is at 2-3% CPU with 60,000+ monthly visitors
33:06 – Debian Bullseye is running on the server
34:38 – Ansible is used to provision the server
38:47 – Deploying a new podcast episode to the site
41:59 – Making sure all tags are properly filled out
43:46 – Another shell script to auto-inject an episode’s length and file size in bytes
46:55 – There’s no secret management because there’s no secrets
47:29 – The code is backed up on GitHub and an external USB HDD
49:54 – DigitalOcean’s built in monitoring and alerting as well as Uptime Robot is being used
52:34 – Best tips? Find tools that you’re genuinely happy using and stick with them
54:39 – You can find Nick on his site, @nickjanetakis on Twitter and nickjj on GitHub

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

A Custom Electronic Medical Record System for an Ophthalmology Clinic

Mon, 06 Sep 2021 00:00:00 -0400

In this episode of Running in Production, Jason Swett goes over building an internal medical record system with Ruby on Rails. It’s hosted on AWS using Kubernetes and it’s been up and running since 2019.

Jason talks about replacing a few 3rd party services with 1 custom solution, using custom generators, embracing PORO, transitioning from Ansible and individual servers to Kubernetes, making safe decisions while learning as you deploy new things and much more.

Topics Include

1:14 – The dream is to replace 9 separate systems with 1 custom solution
3:02 – Deploying on day 1 and what exactly is an EMR?
6:20 – Motivation for choosing Ruby on Rails to build a medical system
10:39 – Infrastructure as code is important
13:36 – The app does scheduling and soon emails will be sent to patients
15:29 – Specific things the app does and how Rails helps Jason accomplish them
21:56 – Custom generators are used for different modules
25:11 – Every 5 seconds certain forms are auto-saved and a few gems that are being used
27:56 – Rails can only help you so much, then you’re kind of on your own
34:10 – Making a distinction between imperative and declarative code
36:36 – Server rendered templates with sprinkles of Javascript (StimulusJS)
43:53 – Looking into using Hotwire Turbo in the distant future
45:41 – Migrating to use Elasticsearch and using it over PostgreSQL’s full text search
47:49 – Using AWS for hosting and landing on using Kubernetes with EKS
52:57 – 5% of Jason’s brain is taken up at all times knowing he’s on-call
55:22 – Switching from individual servers with Ansible to Kubernetes
59:37 – Using eksctl and Helm to move towards infrastructure as code
1:06:12 – The deploy process from development to production with Kubernetes
1:09:53 – Using feature flags and code reviews
1:11:35 – Handling database migrations with a load balanced application
1:15:13 – A strategy for writing Jira tickets
1:17:53 – RDS snapshots are handling database backups
1:21:14 – Monitoring, alerting and the idea of “sharpening the saw”
1:25:47 – Best tips? Make safe decisions and improve skills such as reading documentation
1:30:44 – Check out Jason’s site at https://www.codewithjason.com/

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Games Directory Lets You Sync Your Games and Achievements in 1 Place

Mon, 30 Aug 2021 00:00:00 -0400

In this episode of Running in Production, Vlad Radulescu goes over creating a game directory site with Ruby on Rails. It’s hosted on AWS and has been up and running since 2014.

Vlad talks about having thousands of active users, interfacing with a few game platform APIs, running millions of Sidekiq jobs, storing 10+ billion database records, keeping things as a monolithic app, deploying the web app to 1 server and lots more.

Topics Include

6:30 – Motivation for switching from PHP to Ruby on Rails
7:07 – Figuring out which gaming API to implement first and how their APIs are
10:27 – Reverse engineering undocumented API calls from the Windows Steam client
12:18 – Using a separate database for each gaming platform provider
15:05 – A few useful features of Rails that’s being used and Stimulus Reflex
20:19 – Switching from Webpacker to using Vite
23:14 – Using Slim instead of ERB and a few other gems being used
25:29 – It’s a single monolithic Rails app that’s using namespaces
28:36 – A lot of the time spent developing this app is working with the game APIs
31:00 – Using Sidekiq, a billion (!) gamer activities stored and millions of Sidekiq jobs
34:18 – It’s hosted on a single AWS T2.large instance (4 vCPUs / 16 GB memory)
36:13 – The database is on an R6g.large with 10+ billion records
39:14 – Ansible was used to set up the EC2 instance and it’s running Ubuntu 18.04 LTS
41:13 – A couple of AWS resources being used
43:01 – The process to ship a feature from development to production
44:16 – Switching to using TailwindCSS in a nicely controlled way
47:57 – At the moment the site is free with no desire to make it pay to win
49:55 – Hosting is $1,500 pounds a month and it’s been running for free for 7+ years
51:43 – There’s 7 terabytes of S3 storage and daily database backups
55:59 – 50ish mini servers spread across Heroku’s free tier to bypass API rate limits
59:45 – Being dialed into the ops side of things but there’s room for improvement
1:00:54 – Keeping up with a full date job and a long running passion project
1:02:25 – Best tips? Don’t forget to plan things out
1:05:56 – You can find Vlad on Twitter and Games Directory will be open source soon

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

School Bus Hero Helps School Bus Drivers and Aides Find a Job

Mon, 23 Aug 2021 00:00:00 -0400

In this episode of Running in Production, Dieter Lunn goes over building a job board site for school bus drivers and aides using Ruby on Rails. It’s hosted on DigitalOcean with HatchBox.

Dieter talks about using a bit of StimulusJS to add pins to a map, keeping things simple with a monolithic app, working on the code base with another developer, upgrading to the latest versions on a regular basis and using HatchBox to manage the servers.

Topics Include

3:13 – What the site does and examples of what types of pages it has
5:21 – Motivation for using Ruby on Rails
7:02 – Specific features of Rails and gems being used
11:04 – Using StimulusJS for placing pins on an embedded 2D map
14:02 – One feature lets you get emailed when new positions open at a company
15:40 – It’s a server rendered monolithic Rails app with sprinkles of JS
19:14 – Keeping your gems up to date
23:15 – Using Pundit to manage authorization
25:27 – Keeping up with the latest versions and having an app starter project
27:09 – It’s hosted with HatchBox on DigitalOcean
32:02 – Walking us through developing a feature and deploying it to production
38:07 – Payments are handled outside of the Rails app
39:39 – Automation feels pretty good!
43:45 – Currently Honeybadger sends notifications if the site is down
45:47 – Best tips? Take your time and trust what others have built
48:08 – You can find Dieter on GitHub and on Twitter

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Hitobito Helps You Manage Communities with Complex Group Hierarchies

Mon, 16 Aug 2021 00:00:00 -0400

In this episode of Running in Production, Matthias Viehweger goes over building a service to help organize groups of people. It’s built with Ruby on Rails and is hosted on OpenShift with Kubernetes. It’s been running in production since 2012.

Matthias talks about building a Rails Engine abstraction, creating a multi-repo monolith, using Sphinx for full text search, making the most of Kubernetes / OpenShift and lots more.

Topics Include

6:23 – There’s an open source version of it along with a hosted SAAS app
9:10 – Motivation for using Ruby on Rails and updating from v4 to v6 over time
12:18 – Using Action Mailer and Delayed Job to send and receive emails
16:40 – A few other gems being used in the project
19:25 – 86 models in the core app and 275 extra for a specific Scout site (~70k LOC too)
22:49 – How billing is handled for the hosted SAAS app (it’s through invoices)
24:54 – Handling integrations by exporting data
26:31 – It’s a multi-repo monolithic application, what the “core” and “wagon” are
32:02 – There’s mostly server rendered templates with a bit of JS on the front-end
35:00 – Full text search is handled with the Thinking Sphinx gem
37:09 – There’s a separate MySQL database for each tenant
42:35 – Tech stack run down so far and how memcached is being used
44:58 – The app is set up to use Docker Compose in dev but Matthias doesn’t use it
48:05 – It’s hosted on APPUiO which is a hosted version of OpenShift
50:20 – We’re all YAML engineers, configuring Kubernetes and DB migrations
1:03:04 – Kustomize is being used instead of Helm for templating YAML files
1:05:20 – The Kubernetes related code is in its own git repo
1:08:01 – From developing a feature locally to pushing it to production
1:19:38 – Resource limits are defined in the Kubernetes config files
1:27:40 – Backing up the database with daily Kubernetes cron jobs
1:31:02 – Developers are treated like grown ups when it comes to customer data
1:34:48 – Handling logging and alerting with Prometheus, Grafana and Kubernetes
1:40:51 – Handling DNS with DNSimple and SSL certificates with Let’s Encrypt
1:44:05 – Best tips? Start with a specific thing instead of making a generic thing
1:46:45 – Check out https://hitobito.com/ and their GitHub account, Matthias is also on GitHub, Twitter and he has a site at http://kronn.de/, also Puzzle is the company he works for

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Submotion Helps You Manage Access Control for Your SAAS Subscriptions

Mon, 09 Aug 2021 00:00:00 -0400

In this episode of Running in Production, Kristian Dupont goes over building a SAAS app to manage access control to your existing SAAS subscriptions. It’s built with Koa, Node and React and is hosted on Heroku. It’s been running in production since 2018.

Kristian talks about validating his idea before coding it, really leveraging code linting tools, the challenges of adding a bunch of different SAAS app integrations, using ElephantSQL to host his PostgreSQL database and more.

Topics Include

3:20 – Starting with an empty folder as a sole developer looking to ship an MVP
4:29 – Motivation for using Node
9:19 – Reasons for choosing Koa and Knex with a custom library Kristian wrote
14:57 – Using eslint and the overall power of going all-in with linting
17:50 – The back-end is a RESTful API with React on the front-end
21:43 – Using interesting PostgreSQL features such as triggers
23:30 – Managing the front-end assets with Parcel and using TailwindCSS
30:27 – Fighting for your abstractions and creating React components
32:29 – Adding a bunch of SAAS app integrations was challenging at times
37:57 – Using Jest to run tests but not super happy about it
42:34 – 10k+ lines of code on the front-end and back-end
46:43 – Taxes in the US are fun (not)
49:00 – Mailchimp, Freshping, Datadog and Sentry are being used for various things
52:05 – Redis isn’t being used but the database is quite optimized
55:08 – The web app servers are hosted on Heroku and PostgreSQL is on ElephantSQL
58:45 – Running a Linux VM inside of an M1 Mac for local development
1:02:45 – The deployment process from development to production
1:07:49 – The DB is backed up every hour
1:09:43 – Freshping, Datadog and Slack provide good monitoring and alerting
1:12:09 – Best tips? Using Markdown to store test fixtures
1:15:24 – You can find Kristian on Twitter and his personal site lists all of his socials

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

HorseRecords Lets You Record Everything about Your Horses

Mon, 02 Aug 2021 00:00:00 -0400

In this episode of Running in Production, Andy Ide goes over building a SAAS app for horses with Django and Python. It’s hosted on a single Linode server and has been running in production since May 2021.

Andy talks about learning Django while building his app, using Django Unicorn, creating a Django monolith that’s broken up with Django apps, outsourcing setting up a server, the importance of having a good testing team and more.

Andy was kind enough to give listeners of this podcast 20% off for life if you use promo code RUNNING20 when signing up, that includes signing up for free since it carries over to a paid plan.

Topics Include

6:11 – About 60ish users signed up about a month after launching
8:34 – The process of learning Django while building this app
11:29 – Using the Django admin and a bunch of other useful packages
14:34 – Django Unicorn is being used for interactive bits w/o having to write a lot of JS
18:34 – It’s a Django monolithic with a bunch of Django apps
22:27 – How Django Rest Framework is being used for the reporting part of the app
28:46 – The app is mostly Django rendered templates with sprinkles of JS and Unicorn
34:00 – Stripe is handling payments, their API and docs are great
41:02 – (1) $10 / month Linode server hosts everything with 1 CPU core and 2GB of memory
45:25 – Picking the most recent version of Ubuntu and keeping deploys simple
52:05 – Using Cloudflare for DNS and using their paid plan
56:54 – Running tests locally before deploying it
59:12 – Handling backups with Django DB Backup and Linode’s backup service
1:02:12 – Using Sentry for exception reporting, viewing logs and Uptime Robot
1:08:50 – Best tips? Install Sentry and have a good testing team
1:13:37 – Check out HorseRecords on Facebook and Andy’s Django Blog

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Politico Europe Is a Business to Business News and Data Service

Mon, 26 Jul 2021 00:00:00 -0400

In this episode of Running in Production, Karl Roos goes over building a B2B news and data platform with Rails, Node and Python. It’s hosted on AWS using Elastic Beanstalk and has been up and running since 2014.

Karl talks about writing a Rails API back-end, scraping 400+ sites, executing 500k+ daily jobs, using a bunch of AWS resources, what it’s like dealing with a ~500 GB MySQL database, the importance on taking action and more.

Topics Include

1:55 – What type of application we are talking about here?
4:41 – Switching from PHP to a combo of Rails and Node
8:02 – A few useful Ruby gems that were used to help build the app
9:41 – The Vue front-end is for a customer facing dashboard
14:48 – Where D3.js is being used to render charts and the data pipeline
17:43 – Scraping data from 400+ sites and dealing with edge cases
21:38 – The scraper runs on 10-16 EC2 instances through Elastic Beanstalk
26:04 – Each separate service lives in its own git repository and a bit of Serverless
31:06 – Sticking with the latest stable version of Rails and updating dependencies
33:10 – Sprinkles of Python to glue together a few AWS services and translating languages
36:11 – What it’s like using Elastic Beanstalk and executing 500k+ jobs a day
40:04 – Initial AWS credits helped sway the decision to try out AWS initially
43:47 – How CloudFormation and Terraform are being used
48:59 – All devs can push to production, code reviews, linting and the deploy process
54:21 – Dealing with database migrations and a ~400-500 GB data set
1:01:17 – Getting a local dump of the DB in development, seeding data and secrets
1:06:03 – Database backups are done with the built in RDS snapshots
1:10:19 – Best tips? Less thinking, more doing and learn from your experiments
1:11:35 – Karl is on GitHub and Twitter

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Couchmate Is a Social Chat Platform for Viewers of Live TV

Mon, 19 Jul 2021 00:00:00 -0400

In this episode of Running in Production, Matt Oliver goes over building a chat platform with Akka, Scala and React Native. It’s hosted on DigitalOcean with Kubernetes and has been running in production since 2013.

Matt talks about rewriting his app a few times, handling tens of thousands of nightly TV shows, going all-in with Kubernetes, using Terraform, understanding it’ll take a while to learn things along the way and more.

Topics Include

4:55 – Going from PHP to Scala to Node and back to Scala for the back-end
15:09 – Tens of thousands of channels are created every night
16:39 – What exactly is Akka and the actor model
19:55 – Websockets are being used quite heavily
23:04 – A few Scala libraries that were useful for building this platform
27:31 – There’s a mobile front-end using React Native for Android and iOS
31:30 – Storing and caching TV listings with Gracenote
34:53 – Roughly ~25k lines of Scala and ~11k on the front-end in 2 repos
39:41 – Motivation for using React Native instead of using native languages
45:55 – How to deal with large show TVs with a massive audience
50:28 – The app is basically 3 screens
54:34 – Handling user uploaded gifs and link submissions
1:00:09 – What it was like building out the React Native front-end
1:05:54 – Reasons for picking DigitalOcean and using Kubernetes
1:14:44 – Most of the infrastructure is managed by Terraform
1:16:57 – What it was like to go from not using Kubernetes to going all-in with it
1:24:21 – (3) 2 GB of memory / 2 CPU core servers are running the cluster
1:26:12 – What it’s like developing a new feature and deploying it to production
1:34:28 – The database is backed up on a schedule and before schema changes
1:39:21 – Prometheus, Grafana, Kamon and Sentry are used for metrics and monitoring
1:43:41 – Pingdom is used for an external site monitor
1:44:31 – It took about a month to get confident in using Kubernetes
1:49:29 – Best tips? It’s going to take a while, hang in there
1:53:06 – Writing comments for your future self while learning
1:54:34 – Matt is on Twitter and most socials as @halfmatthalfcat and @couchmatehq

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Managing 40+ Servers in a Data Center at a Medical University

Mon, 12 Jul 2021 00:00:00 -0400

In this episode of Running in Production, Maciej Delmanowski talks about building out a 40+ server / 200+ VM data center with Ansible. It’s hosted on premises, he’s been working on it all since 2007 and started using Ansible in 2013.

Maciej talks about automating everything with Ansible, being a sysadmin for over a decade, how he picked Debian, splitting up a project into 120+ git repos and then back to 1, writing 60k+ lines of YAML, using Linux Containers, maintaining an open source project and more.

Topics Include

2:35 – There’s about 40 physical servers and 200+ virtual machines / containers
4:50 – Transitioning to using Ansible over time
7:40 – A 10+ year friendship stemming from Debian and open source
8:23 – How Ansible’s role and inventory abstractions help manage a lot of VMs
10:35 – How DebOps as a name came into existence and its philosophy on being stable
14:24 – Motivation for choosing and staying with Debian
15:31 – Figuring out what new Ansible roles and playbooks to work on
19:14 – Going from a mono repo to 120+ repos and then back to a mono repo
24:48 – 67,000+ lines of YAML and 40,000+ lines of documentation
26:28 – Setting up a brand new server with Ansible and DebOps hands free in 15 minutes
28:29 – Automatically generating random passwords for services
31:22 – Not having to deal with HIPAA compliance laws and handling student emails
34:01 – Let’s Encrypt is being used on specific publicly accessible servers
34:26 – Breaking down the process of creating a new role from scratch
36:39 – Using Linux Containers in development and rolling things out to production
41:41 – Using dnsmasq in development for fully qualified domain names
43:07 – Dealing with backing everything up
46:02 – Being a maintainer of an open source project that’s extracted from work
49:19 – Best tips? Have 1 role for 1 service and find ways to connect them
51:26 – Check out DebOps, it’s on GitHub, #debops on Libera and Maciej is on Twitter

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Avo Is a Framework for Creating Ruby on Rails Admin Panels

Mon, 05 Jul 2021 00:00:00 -0400

In this episode of Running in Production, Adrian Marin goes over building a Ruby on Rails admin framework. It’s hosted on Heroku and has been available since late 2020.

Adrian talks about building a Rails engine, using Stripe Checkout, building the admin out with Hotwire Turbo, using View Component, creating a very automated CI / CD pipeline to publish the gem and much more.

Topics Include

6:38 – Motivation for using Ruby on Rails
8:57 – The gem does phone home on a 1 hour interval
11:33 – The gem is using Hotwire Turbo but the product site is not
13:10 – Handling billing with Stripe’s Checkout page and a few Rails gems
16:43 – Publishing a Rails engine with its own assets
21:10 – Using Hotwire Turbo to build the admin dashboard
26:30 – What it was like migrating the app from Vue to Turbo
31:49 – Using View Component to help improve Rails partial performance
39:21 – Postgres is being used as the database and Redis for caching
42:30 – What is Hotjar, how it’s being used and extracting features
46:36 – From Notion to Linear to using GitHub to help manage the project
49:44 – Using Heroku for hosting and what delayed job is being used for
54:55 – What it’s like deploying the site and the gem
1:01:19 – Performing database backups? Maybe, maybe not!
1:03:57 – Best tips? Start building and launch as soon as possible
1:06:06 – The gem has an open source version that’s very usable
1:07:03 – Check out https://avohq.io/ and Adrian is on Twitter

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Optidash Is an Image Processing and Optimization API

Mon, 28 Jun 2021 00:00:00 -0400

In this episode of Running in Production, Przemek Matylla talks about building an image processing and optimization API with mostly C, Python and Node. It’s hosted on bare metal servers in a data center and has been running in production since 2019.

Przemek talks about handling 20-50 million+ daily API calls, how they’re using C, image detection techniques, using Lua scripting with nginx, building their own servers in a data center, using boring technology and much more.

Topics Include

3:17 – An average day has about 20 million API calls, busy days have 50m+
4:11 – Breaking down where C, Node and other languages are being used
6:46 – What happens when you upload an image to their API
9:06 – Really figuring out the file type of something that’s been uploaded
11:54 – Dealing with edge cases as they come up but preparing a bit ahead of time
14:45 – Switching from Core ML on Apple hardware to Tensorflow on AMD hardware
19:43 – There’s no framework sitting on top of the Node API server
22:28 – The customer facing web dashboard is using Express, the marketing site is Jekyll
24:41 – They’re mostly B2B so feature requests end up being 1 on 1 calls
25:23 – Handling payments with Stripe and using a Node / Angular app for it
28:04 – Using Lua with nginx for rate limiting, also nginx is their load balancer
31:05 – You can’t go wrong with boring and predictable technology
31:47 – MongoDB, Redis and Elasticsearch are all running on 3 nodes each
32:18 – Having nearly instant access to a ton of data helps figure things out
34:52 – What it was like finding a freelance C developer
35:55 – Sending webhooks out is controlled by a separate Node Bull driven app
38:41 – Dealing with GDPR compliance and storing images on GlusterFS for 1 hour
40:50 – Going with bare metal servers in their own data center over the cloud
44:51 – The servers have 32-256GB of memory and a range of different CPUs
46:34 – Having spare parts and dealing with hardware failures
49:15 – About 50 servers run the latest Ubuntu LTS and are managed with Puppet
51:22 – The deployment process for a number of different services
54:19 – It takes ~30min to replace a drive and every service is tripled up
56:48 – The database servers are replicated and there’s alarms and alerts set up
58:56 – Rate limiting was put in place for limiting API calls to customers
1:01:12 – There’s custom payment rates depending on each customer’s requirements
1:03:06 – Best tips? Over provision like crazy and monitoring lets you sleep at night
1:03:31 – Do what works for you, don’t copy another company because it works for them
1:05:48 – Check out https://optidash.ai, their tech blog and GitHub account

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

CourseMaker Is an Online Course Builder for Technical Makers

Mon, 21 Jun 2021 00:00:00 -0400

In this episode of Running in Production, Chris Samiullah goes over building a video course hosting platform with FastAPI, Strapi and Gatsby. It’s hosted on ECS along with a bunch of AWS resources. It’s been running in production since early 2021.

Chris talks about combining a Gatsby static site with FastAPI, using AWS Fargate, streaming videos with Cloudflare, supporting both Stripe and Paddle, using a bit of Serverless Lambdas and tons more.

Topics Include

3:19 – Hiring outside contractors to tackle setting up the infrastructure
5:05 – Motivation for choosing Strapi, FastAPI, React, Gatsby and more
9:37 – Strapi is being used as a headless CMS for the student watching front-end
10:33 – Going over how the student watching experience is set up
14:20 – Creating a course preview with Fargate using sub-domains and DNS entries
16:35 – A couple Gatsby plugins that helped build the site
18:57 – A couple of Python libraries to help build out the FastAPI back-end
20:54 – Using Cloudflare for streaming the videos and Shaka as the video player
26:39 – Stripe and Paddle are being used to handle payments (including VAT)
37:35 – About 20k lines of Python and 10k lines of front-end / React code
39:53 – Everything is Dockerized, there’s PostgreSQL and an ALB too
43:30 – Using Let’s Encrypt and letting course owners hook up custom domain names
45:05 – Reasons for choosing AWS and ECS (hosting is about $100 to $150 a month)
51:45 – Having a dev (staging) server and production in different regions
53:27 – What it’s like coding things up in development and pushing them to production
58:42 – The dev experience for using AWS Lambdas and creating internal tooling
1:05:59 – On-boarding new authors (doing something that doesn’t scale but it works now)
1:10:33 – A public service announcement about database migrations
1:13:13 – FastAPI has been working very nicely
1:16:41 – Backing up the database and user generated files
1:19:56 – Using Sentry for tracking errors and getting notified over Slack
1:25:35 – Best tips? Pick a hosting infrastructure that fits your project
1:27:44 – Picking between ECS and EKS was left to the contractor
1:30:03 – Check out https://coursemaker.org/, Chris is on Twitter and LinkedIn too

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Statuspal Is a Service for Hosted Status Pages and Monitoring

Mon, 14 Jun 2021 00:00:00 -0400

In this episode of Running in Production, Eduardo Messuti talks about building a status page and monitoring service with Phoenix and Elixir. It’s hosted on DigitalOcean and Heroku. It’s been up and running since 2017.

Eduardo talks about upgrading their code base to use Phoenix contexts, using Oban, creating a notification abstraction, going all-in with EEx rendered templates and a bit of jQuery, using Docker Compose in production and more.

Topics Include

1:59 – What it was like using Elixir back in 2017
4:52 – Elixir handles 500k requests a day and motivation to use Elixir / Phoenix
8:17 – The process to upgrade the code base to use Phoenix contexts
10:02 – Oban is being used to handle background jobs
14:39 – Sending out notifications and webhooks with custom code
16:39 – Using Mix format as part of a CI pipeline with GitHub Actions
19:00 – Breaking out the mail service into its own service
21:03 – About 28k lines of Elixir and 2k lines of JS
23:48 – Server rendered app with EEx vs going the SPA / API route vs Live View
28:45 – Supplying a status page embed widget for customers and custom domains
33:08 – Using nginx as a reverse proxy, ETS for caching along with Cloudflare
35:03 – Hugo is being used to serve the static marketing site
37:11 – Docker is being used in staging and production
38:17 – The mailer is hosted on Heroku, everything else is on DigitalOcean
39:58 – The hardware specs of the 4-5 DigitalOcean servers and what they do
42:27 – They’re running Ubuntu 20.04 LTS and upgraded the servers on the fly
44:13 – The servers are manually provisioned and run Docker Compose
50:52 – Deploying a feature from development to production
53:37 – User logos get uploaded and saved to disk with a Docker volume
54:52 – Using BackupPC for backing up the database, certificates and file uploads
56:40 – DigitalOcean’s monitoring and Pingdom are being used for various health checks
57:38 – Honeybadger and Papertrail for errors + logging, and the Elixir experience
1:00:38 – Best tips? Pick a technology you’re comfortable with and start building
1:01:31 – What it was like hiring Elixir contract workers
1:02:40 – Check out https://statuspal.io/ with promo code PRODUCTION to save 10%

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

QA Wolf Helps You Create Automated Browser Tests as You Use Your Site

Mon, 07 Jun 2021 00:00:00 -0400

In this episode of Running in Production, Jon Perl goes over building a service that automates creating browser tests for your web apps. It’s built with Node and React and is hosted on Netlify, Vercel, Azure Containers and DigitalOcean. It’s been running in production since 2019.

Jon talks about creating an open source tool and turning it into a SAAS app, using VNC and websockets to stream test runner data to the browser, keeping things as a mono repo, moving from Azure Containers to Kubernetes and much more.

Topics Include

3:11 – Going from a self hosted version to self hosted + hosted SAAS app
7:20 – Motivation for using Node and React
8:33 – Docker is being used for the test runners
11:24 – A mono repo with Next.js, GraphQL and Apollo
15:33 – Balancing your unit tests with end to end tests
18:25 – Using noVNC and websockets to link the test runner output to the browser
22:58 – Spinning up separate Docker containers for each test runner
24:52 – Next.js, Webpack, React and Typescript make up the front-end JS
28:37 – Efficiently sending VNC data over websockets
32:21 – Handling usage limits with the API server
35:55 – PostgreSQL is their primary database and they’re using Pipedream
38:01 – Stripe is being used to handle payments and SendGrid to send emails
42:00 – Everything is hosted on a combo of Netlify, Vercel, Azure and DigitalOcean
47:11 – Scaling up and down Azure containers with custom Node scripts
49:06 – Moving to Kubernetes in the future but only after it was deemed worth it
57:08 – The current deployment strategy pre-Kubernetes
1:00:03 – Jest tests are written and the code is linted with ESLint
1:04:06 – Database backups and the front-end is deployed to 2 hosts for redundancy
1:06:11 – Datadog handles monitoring and logging with Slack for notifications
1:09:30 – Keeping your assets fresh in a SPA or a long lived app in a browser tab
1:14:44 – Best tips? Focus on getting something out and learning from it
1:16:32 – Toggling feature flags with ENV variables and database fields
1:18:03 – Building the selector generator was a tricky problem to solve
1:23:06 – Check out https://www.qawolf.com/, the app is on GitHub and you can email Jon

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

ListenAddict Lets You Subscribe to a Person Like You Would to a Show

Mon, 31 May 2021 00:00:00 -0400

In this episode of Running in Production, David Parker talks about building a service to subscribe to people with Ruby on Rails and Sapper. It’s hosted on Heroku ($30 / month) and has been up and running since November 2020.

David talks about creating an API back-end with Rails, trying out Svelte and Sapper, the challenges of scraping names from websites, figuring out and fixing PostgreSQL performance issues, finding a good work / life balance and more.

Topics Include

3:27 – Motivation for using Ruby on Rails and Sapper / Svelte
6:18 – Writing raw SQL queries before translating them to ActiveRecord
7:36 – A few Ruby libraries that helped build this project
9:37 – The challenges around scraping human names from various websites
13:17 – The API Rails back-end is a monolithic app with about 100 endpoints
15:44 – David pulls from 94 different sources to find new content
19:06 – The Engtagger gem helps figure out stop words
20:25 – Why did you go with an API back-end?
21:43 – What Svelte and Sapper are and how they help build UIs
25:34 – How the back-end and front-end communicate with each other
28:29 – ~11k LOC on the back-end and ~5 LOC on the front-end
30:49 – At the moment it’s a free service and there’s no ads too
33:23 – What Sidekiq is being used for in terms of background jobs
35:30 – Mailgun is being used to send all transactional emails
37:34 – How the service might be monetized in the future (it’s $30 / month for hosting now)
40:31 – Break down of Heroku Dynos and add-ons
43:17 – Figuring out how to add new people and also using Cloudflare
45:40 – The deployment process from development to production
49:16 – The database is being backed up through Heroku
52:00 – A features table is in the database to enable features for himself
54:37 – Logentries will send out a notification if something goes down
57:50 – What to look out for and how to fix certain performance issues with PostgreSQL
1:01:09 – An example of when to consider breaking up a wide table into multiple tables
1:05:22 – Best tips? Figure out your work / life balance and try the back-end / front-end split
1:11:21 – Find David on Twitter, https://www.programmingtil.com/ and his YouTube channel

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

PriceTable Mixes in Sales Automation, Project Management and Invoicing

Mon, 24 May 2021 00:00:00 -0400

In this episode of Running in Production, Ege Ersoz goes over building a project management service using Phoenix and Elixir. It runs on Gigalixir along with Heroku and has been up and running since early 2019.

Ege talks about using Elixir for the last few years, upgrading to use Phoenix contexts, building a monolithic app with an API back-end / VueJS front-end, building his own Stripe billing module, what it’s been like using Gigalixir and more.

Topics Include

1:24 – What it was like using Elixir back in early 2019
4:55 – Motivation for using Phoenix and Elixir
6:37 – Phoenix Channels and Elixir GenServers are being used a bit
10:14 – It’s a monolithic app with mostly an API back-end using VueJS on the front-end
13:34 – Going through the process of upgrading to use Phoenix contexts
18:19 – Interesting packages in Ege’s mix file that helped him build his app
23:00 – The front-end assets are managed with Webpack, Vuetify is being used too
26:47 – Separation of concerns was the main reason for making an API back-end
29:43 – Stripe is being used to accept payments with no 3rd party Stripe library
32:27 – PostgreSQL is the primary database and it’s all hosted on Gigalixir
36:38 – The Gigalixir bill is about $200 a month including a staging environment
39:49 – The deployment process from development to production
42:07 – Offloading PDF generation to a micro-service
43:38 – Gigalixir is handling automated database backups
45:44 – UptimeRobot monitors the app and reports back if it’s down
47:05 – Getting rid of subdomains for each user account / tenant and multi-tenancy
51:34 – Moving from Timber to Logflare for application logs
54:26 – Transactional emails are sent using SendGrid
55:57 – Best tips? Having a trustworthy co-founder has been incredibly important
59:12 – Check out https://pricetable.io/ and you can email Ege directly too

Support the Show

This episode does not have a sponsor and this podcast is a labor of love. If you want to support the show, the best way to do it is to purchase one of my courses or suggest one to a friend.

Dive into Docker is a video course that takes you from not knowing what Docker is to being able to confidently use Docker and Docker Compose for your own apps. Long gone are the days of "but it works on my machine!". A bunch of follow along labs are included.

Build a SAAS App with Flask is a video course where we build a real world SAAS app that accepts payments, has a custom admin, includes high test coverage and goes over how to implement and apply 50+ common web app features. There's over 20+ hours of video.

Running in Production

Meteor Cloud Is a Full Service Hosting Solution for Meteor Apps

Topics Include

Links

📄 References

⚙️ Tech Stack

Support the Show

CompanyCam Helps Contractors Document Their Job

Topics Include

Links

📄 References

⚙️ Tech Stack

🛠 Libraries Used

Support the Show

TinyPilotKVM Lets You Remote Control Your Server from Your Browser

Topics Include

Links

📄 References

⚙️ Tech Stack

🛠 Libraries Used

Support the Show

uscreen Is a Platform That Helps Content Creators Build a Business

Topics Include

Links

📄 References

⚙️ Tech Stack

🛠 Libraries Used

Support the Show

Great Question Makes It Easy for Teams to Perform Customer Research

Topics Include

Links

📄 References

⚙️ Tech Stack

🛠 Libraries Used

Support the Show

Robot Accounts AI Provides a System to Let You Categorize Invoices

Topics Include

Links

📄 References

⚙️ Tech Stack

🛠 Libraries Used

Support the Show

Fundraze Is a Flexible Fund Raising and Campaign Management Service

Topics Include

Links

📄 References

⚙️ Tech Stack

🛠 Libraries Used

Support the Show

Running in Production Is a Podcast Where Devs Chat about Tech Stacks

Topics Include

Links

📄 References

⚙️ Tech Stack

🛠 Libraries Used

Support the Show

A Custom Electronic Medical Record System for an Ophthalmology Clinic

Topics Include

Links

📄 References

⚙️ Tech Stack

🛠 Libraries Used

Support the Show

Games Directory Lets You Sync Your Games and Achievements in 1 Place

Topics Include

Links

📄 References

⚙️ Tech Stack

🛠 Libraries Used

Support the Show

School Bus Hero Helps School Bus Drivers and Aides Find a Job

Topics Include

Links

📄 References

⚙️ Tech Stack

🛠 Libraries Used

Support the Show

Hitobito Helps You Manage Communities with Complex Group Hierarchies

Topics Include

Links