Containerization and Kubernetes - Some Clarity

It is impossible to talk about Docker without first exploring containers. Containers solve a critical issue in the life of application development. When developers are writing code they are working on their own local development environment. When they are ready to move that code to production this is where problems arise. The code that worked perfectly on their machine doesn’t work in production. The reasons for this are varied; different operating system, different dependencies, different libraries. 

Containers solved this critical issue of portability allowing you to separate code from the underlying infrastructure it is running on. Developers could package up their application, including all of the bins and libraries it needs to run correctly, into a small container image. In production that container can be run on any computer that has a containerization platform.

Advantages of Containers

In addition to solving the major challenge of portability, containers and container platforms provide many advantages over traditional virtualization. 

Containers have an extremely small footprint. The container just needs its application and a definition of all of the bins and libraries it requires to run. Unlike VMs which each have a complete copy of a guest operating system, container isolation is done on the kernel level without the need for a guest operating system. In addition, libraries can be across containers, so it eliminates the need to have 10 copies of the same library on a server, further saving space. If I have 3 apps all running node and express, I don't have to have 3 instances of node and express, those apps can share those bins and libraries.  Allowing for applications to become encapsulated in self-contained environments allows for quicker deployments, closer parity between development environments, and infinite scalability.

What is Docker?

Docker is currently the most popular container platform. Docker appeared on the market at the right time, and was open source from the beginning, which likely led to its current market domination. 30% of enterprises currently use Docker in their AWS environment and that number continues to grow.

When most people talk about Docker they are talking about Docker Engine, the runtime that allows you to build and run containers. But before you can run a Docker container they must be built, starting with a Docker File. The Docker File defines everything needed to run the image including the OS network specifications, and file locations. Now that you have a Docker file, you can build a Docker Image which is the portable, static component that gets run on the Docker Engine. And if you don’t want to start from scratch Docker even has a service called Docker Hub, where you can store and share images. 

The Need for Orchestration Systems

While Docker provided an open standard for packaging and distributing containerized applications, there arose a new problem. How would all of these containers be coordinated and scheduled? How do you seamlessly upgrade an application without any interruption of service? How do you monitor the health of an application, know when something goes wrong and seamlessly restart it?

Solutions for orchestrating containers soon emerged. Kubernetes, Mesos, and Docker Swarm are some of the more popular options for providing an abstraction to make a cluster of machines behave like one big machine, which is vital in a large-scale environment.

When most people talk about “Kubernetes vs. Docker,” what they really mean is “Kubernetes vs. Docker Swarm.” The latter is Docker’s own native clustering solution for Docker containers, which has the advantage of being tightly integrated into the ecosystem of Docker, and uses its own API. Like most schedulers, Docker Swarm provides a way to administer a large number of containers spread across clusters of servers. Its filtering and scheduling system enables the selection of optimal nodes in a cluster to deploy containers.

Kubernetes is the container orchestrator that was developed at Google which has been donated to the CNCF and is now open source. It has the advantage of leveraging Google’s years of expertise in container management. It is a comprehensive system for automating deployment, scheduling and scaling of containerized applications, and supports many containerization tools such as Docker.

For now, Kubernetes is the market leader and the standardized means of orchestrating containers and deploying distributed applications. Kubernetes can be run on a public cloud service or on-premises, is highly modular, open source, and has a vibrant community. Companies of all sizes are investing into it, and many cloud providers offer Kubernetes as a service. Sumo Logic provides support for all orchestration technologies, including Kubernetes-powered applications.

How does Kubernetes work?

It is easy to get lost in the details of Kubernetes, but at the end of the day, what Kubernetes is doing is pretty simple. Cheryl Hung of the CNCF describes Kubernetes as a control loop. Declare how you want your system to look (3 copies of container image a and 2 copies of container image b) and Kubernetes makes that happen. Kubernetes compares the desired state to the actual state, and if they aren’t the same, it takes steps to correct it. 

Kubernetes architecture and components

Kubernetes is made up many components that do not know are care about each other. The components all talk to each other through the API server. Each of these components operates its own function and then exposes metrics, that we can collect for monitoring later on. We can break down the components into three main parts.

1. The Control Plane - The Master.
2. Nodes - Where pods get scheduled.
3. Pods - Holds containers

The Control Plane - The Master Node

The control plane is the orchestrator. Kubernetes is an orchestration platform, and the control plane facilitates that orchestration. There are multiple components in the control plane that help facilitate that orchestration. Etcd for storage, the API server for communication between components, the scheduler which decides which nodes pods should run on, and the controller manager, responsible for checking the current state against the desired state. 

Nodes

Nodes make up the collective compute power of the Kubernetes cluster. This is where containers actually get deployed to run. Nodes are the physical infrastructure that your application runs on, the server of VMs in your environment. 

Pods

Pods are the lowest level resource in the Kubernetes cluster. A pod is made up of one or more containers, but most commonly just a single container. When defining your cluster, limits are set for pods which define what resources, CPU and memory, they need to run. The scheduler uses this definition to decide on which nodes to place the pods. If there is more than one container in a pod, it is difficult to estimate the required resources and the scheduler will not be able to appropriately place pods. 

How Does Kubernetes Relate to Docker?

Kubernetes and Docker are both comprehensive de-facto solutions to intelligently manage containerized applications and provide powerful capabilities, and from this some confusion has emerged. “Kubernetes” is now sometimes used as a shorthand for an entire container environment based on Kubernetes. In reality, they are not directly comparable, have different roots, and solve for different things.

Docker is a platform and tool for building, distributing, and running Docker containers. It offers its own native clustering tool that can be used to orchestrate and schedule containers on machine clusters. Kubernetes is a container orchestration system for Docker containers that is more extensive than Docker Swarm and is meant to coordinate clusters of nodes at scale in production in an efficient manner. It works around the concept of pods, which are scheduling units (and can contain one or more containers) in the Kubernetes ecosystem, and they are distributed among nodes to provide high availability. One can easily run a Docker build on a Kubernetes cluster, but Kubernetes itself is not a complete solution and is meant to include custom plugins.

Kubernetes and Docker are both fundamentally different technologies but they work very well together, and both facilitate the management and deployment of containers in a distributed architecture.

Can you use Docker without Kubernetes?

Docker is commonly used without Kubernetes, in fact this is the norm. While Kubernetes offers many benefits, it is notoriously complex and there are many scenarios where the overhead of spinning up Kubernetes is unnecessary or unwanted. 

In development environments it is common to use Docker without a container orchestrator like Kubernetes. In production environments often the benefits of using a container orchestrator do not outweigh the cost of added complexity. Additionally, many public cloud services like AWS, GCP, and Azure provide some orchestration capabilities making the tradeoff of the added complexity unnecessary. 

Can you use Kubernetes without Docker?

As Kubernetes is a container orchestrator, it needs a container runtime in order to orchestrate. Kubernetes is most commonly used with Docker, but it can also be used with any container runtime. RunC, cri-o, Containerd are other container runtimes that you can deploy with Kubernetes.  The Cloud Native Computing Foundation (CNCF) maintains a listing of endorsed container runtimes on their ecosystem landscape page and Kubernetes documentation provides specific instructions for getting set up using ContainerD and CRI-O. 

Building a web API with ASP.NET Core and MongoDB 🌯

This tutorial creates a web API that performs Create, Read, Update, and Delete (CRUD) operations on a MongoDB NoSQL database.

In this tutorial, you learn how to:

• Configure MongoDB
• Create a MongoDB database
• Define a MongoDB collection and schema
• Perform MongoDB CRUD operations from a web API
• Customize JSON serialization

View or download sample code (how to download)

Prerequisites

• Visual Studio
• Visual Studio Code
• Visual Studio for Mac

• .NET Core SDK 2.2
• Visual Studio 2019 with the ASP.NET and web development workload
• MongoDB
• I suggest Visual Studio Code

Configure MongoDB

If using Windows, MongoDB is installed at C:\Program Files\MongoDB by default. Add C:\Program Files\MongoDB\Server\\bin to the Path environment variable. This change enables MongoDB access from anywhere on your development machine.

Use the mongo Shell in the following steps to create a database, make collections, and store documents. For more information on mongo Shell commands, see Working with the mongo Shell.

1. Choose a directory on your development machine for storing the data. For example, C:\BooksData on Windows. Create the directory if it doesn't exist. The mongo Shell doesn't create new directories.
2. Open a command shell. Run the following command to connect to MongoDB on default port 27017. Remember to replace  with the directory you chose in the previous step.
   console

   mongod --dbpath 
   

3. Open another command shell instance. Connect to the default test database by running the following command:
   console

   mongo
   

4. Run the following in a command shell:
   console

   use BookstoreDb
   

   If it doesn't already exist, a database named BookstoreDb is created. If the database does exist, its connection is opened for transactions.
5. Create a Books collection using following command:
   console

   db.createCollection('Books')
   

   The following result is displayed:
   console

   { "ok" : 1 }
   

6. Define a schema for the Books collection and insert two documents using the following command:
   console

   db.Books.insertMany([{'Name':'Design Patterns','Price':54.93,'Category':'Computers','Author':'Ralph Johnson'}, {'Name':'Clean Code','Price':43.15,'Category':'Computers','Author':'Robert C. Martin'}])
   

   The following result is displayed:
   console

   {
     "acknowledged" : true,
     "insertedIds" : [
       ObjectId("5bfd996f7b8e48dc15ff215d"),
       ObjectId("5bfd996f7b8e48dc15ff215e")
     ]
   }
   

    Note

   The ID's shown in this article will not match the IDs when you run this sample.
7. View the documents in the database using the following command:
   console

   db.Books.find({}).pretty()
   

   The following result is displayed:
   console

   {
     "_id" : ObjectId("5bfd996f7b8e48dc15ff215d"),
     "Name" : "Design Patterns",
     "Price" : 54.93,
     "Category" : "Computers",
     "Author" : "Ralph Johnson"
   }
   {
     "_id" : ObjectId("5bfd996f7b8e48dc15ff215e"),
     "Name" : "Clean Code",
     "Price" : 43.15,
     "Category" : "Computers",
     "Author" : "Robert C. Martin"
   }
   

   The schema adds an autogenerated _id property of type ObjectId for each document.

The database is ready. You can start creating the ASP.NET Core web API.

Create the ASP.NET Core web API project

• Visual Studio
• Visual Studio Code
• Visual Studio for Mac

1. Go to File > New > Project.
2. Select the ASP.NET Core Web Application project type, and select Next.
3. Name the project BooksApi, and select Create.
4. Select the .NET Core target framework and ASP.NET Core 2.2. Select the API project template, and select Create.
5. Visit the NuGet Gallery: MongoDB.Driver to determine the latest stable version of the .NET driver for MongoDB. In the Package Manager Console window, navigate to the project root. Run the following command to install the .NET driver for MongoDB:
   PowerShell

   Install-Package MongoDB.Driver -Version {VERSION}
   

Add an entity model

1. Add a Models directory to the project root.
2. Add a Book class to the Models directory with the following code:
   C#

   using MongoDB.Bson;
   using MongoDB.Bson.Serialization.Attributes;
   
   namespace BooksApi.Models
   {
       public class Book
       {
           [BsonId]
           [BsonRepresentation(BsonType.ObjectId)]
           public string Id { get; set; }
   
           [BsonElement("Name")]
           public string BookName { get; set; }
   
           public decimal Price { get; set; }
   
           public string Category { get; set; }
   
           public string Author { get; set; }
       }
   }
   

   In the preceding class, the Id property:
   • Is required for mapping the Common Language Runtime (CLR) object to the MongoDB collection.
   • Is annotated with [BsonId] to designate this property as the document's primary key.
   • Is annotated with [BsonRepresentation(BsonType.ObjectId)] to allow passing the parameter as type string instead of an ObjectId structure. Mongo handles the conversion from string to ObjectId.
   The BookName property is annotated with the [BsonElement] attribute. The attribute's value of Name represents the property name in the MongoDB collection.

Add a configuration model

1. Add the following database configuration values to appsettings.json:
   JSON

   
   
   {
     "BookstoreDatabaseSettings": {
       "BooksCollectionName": "Books",
       "ConnectionString": "mongodb://localhost:27017",
       "DatabaseName": "BookstoreDb"
     },
     "Logging": {
       "IncludeScopes": false,
       "Debug": {
         "LogLevel": {
           "Default": "Warning"
         }
       },
       "Console": {
         "LogLevel": {
           "Default": "Warning"
         }
       }
     }
   }
   

2. Add a BookstoreDatabaseSettings.cs file to the Models directory with the following code:
   C#

   namespace BooksApi.Models
   {
       public class BookstoreDatabaseSettings : IBookstoreDatabaseSettings
       {
           public string BooksCollectionName { get; set; }
           public string ConnectionString { get; set; }
           public string DatabaseName { get; set; }
       }
   
       public interface IBookstoreDatabaseSettings
       {
           string BooksCollectionName { get; set; }
           string ConnectionString { get; set; }
           string DatabaseName { get; set; }
       }
   }
   

   The preceding BookstoreDatabaseSettings class is used to store the appsettings.json file's BookstoreDatabaseSettings property values. The JSON and C# property names are named identically to ease the mapping process.
3. Add the following highlighted code to Startup.ConfigureServices:
   C#

   
   
   public void ConfigureServices(IServiceCollection services)
   {
       services.Configure(
           Configuration.GetSection(nameof(BookstoreDatabaseSettings)));
   
       services.AddSingleton(sp =>
           sp.GetRequiredService>().Value);
   
       services.AddMvc()
               .SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
   }
   

   In the preceding code:
   • The configuration instance to which the appsettings.json file's BookstoreDatabaseSettings section binds is registered in the Dependency Injection (DI) container. For example, a BookstoreDatabaseSettings object's ConnectionString property is populated with the BookstoreDatabaseSettings:ConnectionString property in appsettings.json.
   • The IBookstoreDatabaseSettings interface is registered in DI with a singleton service lifetime. When injected, the interface instance resolves to a BookstoreDatabaseSettings object.
4. Add the following code to the top of Startup.cs to resolve the BookstoreDatabaseSettings and IBookstoreDatabaseSettings references:
   C#

   using BooksApi.Models;
   

Add a CRUD operations service

1. Add a Services directory to the project root.
2. Add a BookService class to the Services directory with the following code:
   C#

   using BooksApi.Models;
   using MongoDB.Driver;
   using System.Collections.Generic;
   using System.Linq;
   
   namespace BooksApi.Services
   {
       public class BookService
       {
           private readonly IMongoCollection _books;
   
           public BookService(IBookstoreDatabaseSettings settings)
           {
               var client = new MongoClient(settings.ConnectionString);
               var database = client.GetDatabase(settings.DatabaseName);
   
               _books = database.GetCollection(settings.BooksCollectionName);
           }
   
           public List Get() =>
               _books.Find(book => true).ToList();
   
           public Book Get(string id) =>
               _books.Find(book => book.Id == id).FirstOrDefault();
   
           public Book Create(Book book)
           {
               _books.InsertOne(book);
               return book;
           }
   
           public void Update(string id, Book bookIn) =>
               _books.ReplaceOne(book => book.Id == id, bookIn);
   
           public void Remove(Book bookIn) =>
               _books.DeleteOne(book => book.Id == bookIn.Id);
   
           public void Remove(string id) => 
               _books.DeleteOne(book => book.Id == id);
       }
   }
   

   In the preceding code, an IBookstoreDatabaseSettings instance is retrieved from DI via constructor injection. This technique provides access to the appsettings.json configuration values that were added in the Add a configuration model section.
3. Add the following highlighted code to Startup.ConfigureServices:
   C#

   
   
   public void ConfigureServices(IServiceCollection services)
   {
       services.Configure(
           Configuration.GetSection(nameof(BookstoreDatabaseSettings)));
   
       services.AddSingleton(sp =>
           sp.GetRequiredService>().Value);
   
       services.AddSingleton();
   
       services.AddMvc()
               .SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
   }
   

   In the preceding code, the BookService class is registered with DI to support constructor injection in consuming classes. The singleton service lifetime is most appropriate because BookService takes a direct dependency on MongoClient. Per the official Mongo Client reuse guidelines, MongoClient should be registered in DI with a singleton service lifetime.
4. Add the following code to the top of Startup.cs to resolve the BookService reference:
   C#

   using BooksApi.Services;
   

The BookService class uses the following MongoDB.Driver members to perform CRUD operations against the database:

• MongoClient – Reads the server instance for performing database operations. The constructor of this class is provided the MongoDB connection string:
  C#

  
  
  public BookService(IBookstoreDatabaseSettings settings)
  {
      var client = new MongoClient(settings.ConnectionString);
      var database = client.GetDatabase(settings.DatabaseName);
  
      _books = database.GetCollection(settings.BooksCollectionName);
  }
  

• IMongoDatabase – Represents the Mongo database for performing operations. This tutorial uses the generic GetCollection(collection) method on the interface to gain access to data in a specific collection. Perform CRUD operations against the collection after this method is called. In the GetCollection(collection) method call:
  • collection represents the collection name.
  • TDocument represents the CLR object type stored in the collection.

GetCollection(collection) returns a MongoCollection object representing the collection. In this tutorial, the following methods are invoked on the collection:

• DeleteOne – Deletes a single document matching the provided search criteria.
• Find – Returns all documents in the collection matching the provided search criteria.
• InsertOne – Inserts the provided object as a new document in the collection.
• ReplaceOne – Replaces the single document matching the provided search criteria with the provided object.

Add a controller

Add a BooksController class to the Controllers directory with the following code:

C#

using BooksApi.Models;
using BooksApi.Services;
using Microsoft.AspNetCore.Mvc;
using System.Collections.Generic;

namespace BooksApi.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class BooksController : ControllerBase
    {
        private readonly BookService _bookService;

        public BooksController(BookService bookService)
        {
            _bookService = bookService;
        }

        [HttpGet]
        public ActionResult> Get() =>
            _bookService.Get();

        [HttpGet("{id:length(24)}", Name = "GetBook")]
        public ActionResult Get(string id)
        {
            var book = _bookService.Get(id);

            if (book == null)
            {
                return NotFound();
            }

            return book;
        }

        [HttpPost]
        public ActionResult Create(Book book)
        {
            _bookService.Create(book);

            return CreatedAtRoute("GetBook", new { id = book.Id.ToString() }, book);
        }

        [HttpPut("{id:length(24)}")]
        public IActionResult Update(string id, Book bookIn)
        {
            var book = _bookService.Get(id);

            if (book == null)
            {
                return NotFound();
            }

            _bookService.Update(id, bookIn);

            return NoContent();
        }

        [HttpDelete("{id:length(24)}")]
        public IActionResult Delete(string id)
        {
            var book = _bookService.Get(id);

            if (book == null)
            {
                return NotFound();
            }

            _bookService.Remove(book.Id);

            return NoContent();
        }
    }
}

The preceding web API controller:

• Uses the BookService class to perform CRUD operations.
• Contains action methods to support GET, POST, PUT, and DELETE HTTP requests.
• Calls CreatedAtRoute in the Create action method to return an HTTP 201 response. Status code 201 is the standard response for an HTTP POST method that creates a new resource on the server. CreatedAtRoute also adds a Location header to the response. The Location header specifies the URI of the newly created book.

Test the web API

1. Build and run the app.
2. Navigate to http://localhost:/api/books to test the controller's parameterless Get action method. The following JSON response is displayed:
   JSON

   [
     {
       "id":"5bfd996f7b8e48dc15ff215d",
       "bookName":"Design Patterns",
       "price":54.93,
       "category":"Computers",
       "author":"Ralph Johnson"
     },
     {
       "id":"5bfd996f7b8e48dc15ff215e",
       "bookName":"Clean Code",
       "price":43.15,
       "category":"Computers",
       "author":"Robert C. Martin"
     }
   ]
   

3. Navigate to http://localhost:/api/books/{id here} to test the controller's overloaded Get action method. The following JSON response is displayed:
   JSON

   {
     "id":"{ID}",
     "bookName":"Clean Code",
     "price":43.15,
     "category":"Computers",
     "author":"Robert C. Martin"
   }
   

Configure JSON serialization options

There are two details to change about the JSON responses returned in the Test the web API section:

• The property names' default camel casing should be changed to match the Pascal casing of the CLR object's property names.
• The bookName property should be returned as Name.

To satisfy the preceding requirements, make the following changes:

1. In Startup.ConfigureServices, chain the following highlighted code on to the AddMvc method call:
   C#

   
   
   public void ConfigureServices(IServiceCollection services)
   {
       services.Configure(
           Configuration.GetSection(nameof(BookstoreDatabaseSettings)));
   
       services.AddSingleton(sp =>
           sp.GetRequiredService>().Value);
   
       services.AddSingleton();
   
       services.AddMvc()
               .AddJsonOptions(options => options.UseMemberCasing())
               .SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
   }
   

   With the preceding change, property names in the web API's serialized JSON response match their corresponding property names in the CLR object type. For example, the Book class's Author property serializes as Author.
2. In Models/Book.cs, annotate the BookName property with the following [JsonProperty] attribute:
   C#

   
   
   [BsonElement("Name")]
   [JsonProperty("Name")]
   public string BookName { get; set; }
   

   The [JsonProperty] attribute's value of Name represents the property name in the web API's serialized JSON response.
3. Add the following code to the top of Models/Book.cs to resolve the [JsonProperty] attribute reference:
   C#

   using Newtonsoft.Json;
   

4. Repeat the steps defined in the Test the web API section. Notice the difference in JSON property names.

Happy H@X0ring!

🌯🌯🌯🌯🌯🌯🌯🌯🌯