Personal Cloud Storage System with Docker
Personal Cloud Storage System with Docker
Introduction
In an era dominated by data, managing personal files across multiple devices is a common challenge. While commercial cloud services offer convenience, they often come with privacy concerns and subscription costs. This project documents the creation of a self-hosted, secure, and performant personal cloud storage system using a multi-container Dockerized application.
This document outlines the entire development lifecycle of the project, broken down into three phases. Part 1 (DA-1) covers the foundational setup of a persistent storage system with a basic Flask backend and HTML frontend. Part 2 (DA-2) details the migration to a modern React frontend, introducing a multi-stage Docker build and a visual storage analysis feature. Finally, we focus on production-hardening, security, and professional UI/UX by implementing Material-UI, drag-and-drop uploads, non-root containers, and publishing the final images to Docker Hub for versioning and deployment. Part 3 (DA-3) is where we focus on Documentation and Demo of our project.
Objectives of Part 1 (DA-1): Foundational Containerization
To create a Python Flask API backend with endpoints for file upload, download, and listing.
To build a simple HTML, CSS, and vanilla JavaScript frontend to interact with the API.
To containerize both the frontend (using Nginx) and the backend (using Python).
To use Docker Compose to orchestrate both containers.
To establish a custom Docker network (cloudnet) for secure inter-container communication.
To create and attach a named Docker volume (file_storage) to the backend for persistent data storage.
Objectives of Part 2 (DA-2): Modern Frontend & Analytic
To migrate the entire frontend from static HTML/JS to a modern, dynamic React application.
To implement a multi-stage Dockerfile for the frontend, resulting in a small, optimized Nginx image.
To add a new /analysis endpoint to the backend API to calculate storage statistics (total files, size, and file types).
To integrate chart.js into the React frontend to display the storage analysis as a visual pie chart.
To resolve Cross-Origin Resource Sharing (CORS) issues between the React frontend and Flask backend.
To enhance the React frontend's user experience by integrating a professional component library (Material-UI).
To implement user-friendly features like drag-and-drop file uploads (using react-dropzone) and a visual upload progress bar.
To secure the backend container by creating and running the application as a non-root user.
To resolve the HEALTHCHECK dependency in the backend Dockerfile by installing curl in the base image.
To tag the final, production-ready frontend and backend images and push them to Docker Hub.
Objectives of Part 3 (DA-3): Documentation & Demo
We make a report, summarizing the project with the software tools and modules which have been used in this project.
Giving a brief Demo of our project.
Name of the containers involved and the download links
The final application is built using custom Dockerfiles, but they are based on the following official images downloaded from Docker Hub:
python:3.9-alpine (Base image for our backend container)
Link: https://hub.docker.com/_/python
node:18-alpine (Base image for the build stage of our frontend container)
Link: https://hub.docker.com/_/node
nginx:1.25-alpine (Base image for the final stage of our frontend container)
Link: https://hub.docker.com/_/nginx
Name of the other software involved along with the purpose
Docker & Docker Compose: To build, run, and manage the multi-container application.
Flask (Python): Backend API framework to handle file logic.
React (JavaScript): Frontend library for building the dynamic user interface.
Material-UI (MUI): React component library for a polished, professional UI.
Chart.js & React-Chartjs-2: To render the storage analysis pie chart.
React-Dropzone: To enable drag-and-drop file upload functionality.
Axios: JavaScript library to handle API requests between the frontend and backend.
Visual Studio Code: The IDE used for all code development.
Docker Hub: Cloud-based container registry to store and version our final images.
Blogger: The platform used for this documentation.
Overall architecture of all your three DAs
Architecture Diagram -
Architecture Description -
This project's architecture is a classic client-server model, fully containerized and orchestrated by Docker Compose. The system is composed of two primary, isolated services: cloud_frontend and cloud_backend, which communicate over a private bridge network named cloudnet. This isolation ensures the backend's API and file system are never directly exposed to the internet.
The cloud_frontend is an nginx:alpine container serving a highly optimized, static React application. This image is built using a multi-stage process: a node:18-alpine container first compiles the React code, and the resulting static files are then copied into the final, lightweight Nginx image. The user accesses this UI via their browser, which connects to the host's port 8888 (which is mapped to the container's port 80).
The cloud_backend is a secure python:3.9-alpine container running a Flask API. It's responsible for all business logic: processing file uploads, serving downloads, and performing storage analysis. All uploaded files are written to the /data directory inside the container, which is a mount point for the persistent file_storage Docker volume. This crucial design ensures that all user data is safely stored on the host machine, completely independent of the container's lifecycle.
Procedure - Part 1: Foundational Containerization
Project Structure: Created the root directory personal-cloud-storage/ with backend/ and frontend/ subdirectories.
Backend (Flask): Created backend/app.py with basic Flask routes (/upload, /files, /download). Also created backend/requirements.txt with flask and a backend/Dockerfile.
Frontend (HTML): Created frontend/index.html (for UI), style.css (for styling), and script.js (for API calls using fetch).
Frontend Dockerfile: Created frontend/Dockerfile to copy the static files into an nginx container.
Docker Compose: Created the docker-compose.yml file, defining the frontend and backend services, the cloudnet network, and the file_storage volume.
Build & Run: Ran docker-compose up --build.
Result: Accessed http://localhost:8080 and saw the basic UI.
Test: Uploaded a file and saw it persist in the list after a docker-compose down and docker-compose up.
Procedure - Part 2: Modern Frontend & Analytics
Re-init Frontend: Deleted the contents of the frontend/ folder and re-initialized it as a React app using npx create-react-app .
.
(Screenshot : creating a react app by using command)
Install Dependencies: Ran npm install axios chart.js react-chartjs-2 inside the frontend/ directory.
Develop Components: Rewrote the UI into React components: App.js, FileUpload.js, FileList.js, and StorageAnalysis.js.
Update Backend: Modified backend/app.py to add flask-cors to handle requests from the React app. Added the /analysis endpoint, which calculates file counts, total size, and file type distribution.
Multi-stage Dockerfile: Replaced the frontend/Dockerfile with a multi-stage build file. Stage 1 used node:18-alpine to build the React app. Stage 2 copied the build output into nginx:alpine.
Build & Run: Ran docker-compose up --build. This time, the build process for the frontend was much longer as it compiled the React app.
Result: Accessed the new UI at http://localhost:8888 (after changing the port to avoid conflicts).
(Screenshot: The new React UI, looking cleaner than the HTML version)
Test Analysis: Uploaded several different file types (.txt, .jpg, .pdf) and clicked the "Get Analysis" button.
(Screenshot: The pie chart successfully rendering and showing the file type breakdown)
Procedure - Part 3: Security, UX Polish & Deployment
Install UX Dependencies: Ran npm install @mui/material @emotion/react @emotion/styled react-dropzone in the frontend/ directory.
Refactor with MUI: Modified all React components (App.js, FileUpload.js, etc.) to use components from Material-UI (<Button>, <Grid>, <Paper>, <LinearProgress>), replacing the basic HTML elements.
Implement Drag-and-Drop: Used the useDropzone hook in FileUpload.js to create a file dropzone.
Add Progress Bar: Modified the axios.post call in FileUpload.js to include the onUploadProgress config, which updates the state of a <LinearProgress> MUI component.
Secure Backend Image:
Modified backend/Dockerfile to add RUN apk add --no-cache curl to fix the health check.
Added RUN addgroup -S appuser && adduser -S appuser -G appuser to create a non-root user.
Added USER appuser to switch to this user before running the app.
Rebuild Application: Ran docker-compose up --build --force-recreate. Verified that both containers started automatically (health check now works) and the app was functional.
Push to Docker Hub:
Logged in via terminal: docker login
Tagged both images:
docker tag personal-cloud-storage-backend overhorizon12/cloudstorage:backend-latest
docker tag personal-cloud-storage-frontend overhorizon12/cloudstorage:frontend-latest
Pushed both images:
docker push overhorizon12/cloudstorage:backend-latest
docker push overhorizon12/cloudstorage:frontend-latest
(Screenshot: My Docker Hub repository page showing the new cloudstorage repository with the two tags)
What modification is done in the containers after downloading?
We did not just download and run containers; we built highly customized images based on official ones.
python:3.9-alpine (Backend):
Added Packages: Installed curl using the apk package manager.
Added Security: Created a dedicated, non-privileged user and group (appuser) and set the container to run as this user.
Added Code: Copied our app.py and other files into the /app directory.
Installed Dependencies: Ran pip install to add flask and flask-cors.
Configured Health: Added a HEALTHCHECK instruction to ensure the Flask app is running before other containers depend on it.
node:18-alpine (Frontend Build Stage):
Added Code: Copied package.json and our src/ and public/ directories.
Installed Dependencies: Ran npm install to add React, Axios, MUI, Chart.js, and react-dropzone.
Compiled Code: Ran npm run build to compile all JavaScript and CSS into an optimized static build/ folder.
nginx:1.25-alpine (Frontend Server Stage):
Copied Build Artifacts: Copied only the static build/ folder from the node stage into the Nginx web root (/usr/share/nginx/html).
Replaced Configuration: Replaced the default nginx.conf with a custom one to correctly handle React's single-page application (SPA) routing.
Github link / dockerhub link of your modified containers
What are the outcomes of your DA?
A fully functional, persistent cloud storage web application that can be deployed on any machine with Docker.
A secure, multi-container architecture where the backend and database are not exposed, and the backend process runs as a non-root user.
A highly optimized, modern, and user-friendly React frontend with features matching professional cloud services (drag-and-drop, progress bars).
Production-ready, versioned Docker images published on Docker Hub, which enables easy updates, rollbacks, and integration with automated CI/CD pipelines.
Conclusion
This project was a comprehensive journey through the entire lifecycle of a modern web application. It successfully demonstrates how Docker and Docker Compose can be used to build, orchestrate, and secure complex, multi-service applications.
We evolved a simple proof-of-concept (Part 1) into a fully-featured React application (Part 2), and finally hardened it for production by focusing on security, user experience, and deployment (Part 3). The final system not only meets all functional requirements but is also efficient, secure, and ready for automated deployment, showcasing the power of a modern DevOps toolchain.
References
Official Docker Hub Image:
pythonLink:https://hub.docker.com/_/pythonOfficial Docker Hub Image:
nodeLink:https://hub.docker.com/_/nodeOfficial Docker Hub Image:
nginxLink:https://hub.docker.com/_/nginxIITB Docker Tutorial (YouTube) Link: https://spoken-tutorial.org/tutorial-search/?search_foss=Docker&search_language=English
Developer Documentation
Stack Overflow Community:
https://stackoverflow.comMedium Developer Articles:
https://medium.com
Acknowledgement
My sincere thanks to VIT, SCOPE, for designing the 'Cloud Computing' course for the Fall Semester 2025-2026. This project was a core part of the curriculum and provided the structure and motivation for this DA.I would also like to express my gratitude to my course instructor, [Dr. T. Subulakshmi], for their valuable guidance and support throughout the semester. Finally, I want to thank my friends and family for their encouragement and the broader developer community for providing a wealth of public knowledge that was essential to this project's success.
Other Acknowledgements: I would also like to thank my friends, family, and the developer communities on Stack Overflow and Medium for their indirect support and for providing a wealth of public knowledge.
Comments
Post a Comment