Day 3

As we planned to use the same machine and add a new storage device to upgrade the machine capacity for the MVP, we have made a major change in the base structure. We are removing the IP simulation in the node. Now, we are not going to use a multiport structure for MVP, and in further advancement, we will use nodes based on different geolocations.

Key Changes

1. Use Docker Volumes for Persistent Storage:

   • Each node will have its unique volume for storing data.

   • This ensures that data persists even if the container is restarted or recreated.

2. Drop IP Pool Allocation for Local Setup:

   • Nodes can be identified by their node_id.

   • The Docker bridge network can communicate with internal DNS (container_name) or ports.

3. Volume Management:

   • Dynamically create a volume for each node during registration.

   • Map the volume to the container for data storage.

In our development scenario, we have to use a port pool to centrally manage the port allocation of the node.

Using a pool of ports to assign and track them for Docker containers dynamically is a great approach to ensure that:

1. Ports are managed centrally.

2. There are no conflicts in port allocation.

3. You can easily look up which port is assigned to which node.

Implementation Outline

1. Create a Port Pool:

   • Define a range of ports (e.g., 8000-9000) available for allocation.

   • Store this range in Redis as a list or hash.

2. Allocate Ports Dynamically:

   • When a new node is created, check for the next available port in the pool.

   • Mark the port as "used" (e.g., update its status in Redis).

3. Track Port Usage:

   • Use Redis to store a mapping of node IDs to assigned ports.

   • Example structure:

HASH: node_ports
  node1 -> 8000
  node2 -> 8001
SET: available_ports
  8002, 8003, ..., 9000

1. Release Ports:

   • When a node is removed or stopped, mark the port as "available" again.

2. Integrate Port Allocation in the Go Code:

   • Retrieve an available port from Redis.

   • Assign it when creating the Docker container.