Initial commit
Some checks failed
Build and Publish Docker Images / build (server.Dockerfile, ${{ vars.IMAGE_NAME_SERVER }}) (push) Has been cancelled
Build and Publish Docker Images / build (server.Dockerfile, ${{ vars.IMAGE_NAME_WORKER }}) (push) Has been cancelled
Build and Publish Docker Images / build (worker.Dockerfile, ${{ vars.IMAGE_NAME_SERVER }}) (push) Has been cancelled
Build and Publish Docker Images / build (worker.Dockerfile, ${{ vars.IMAGE_NAME_WORKER }}) (push) Has been cancelled
Build and Publish Docker Images / setup (push) Has been cancelled
Some checks failed
Build and Publish Docker Images / build (server.Dockerfile, ${{ vars.IMAGE_NAME_SERVER }}) (push) Has been cancelled
Build and Publish Docker Images / build (server.Dockerfile, ${{ vars.IMAGE_NAME_WORKER }}) (push) Has been cancelled
Build and Publish Docker Images / build (worker.Dockerfile, ${{ vars.IMAGE_NAME_SERVER }}) (push) Has been cancelled
Build and Publish Docker Images / build (worker.Dockerfile, ${{ vars.IMAGE_NAME_WORKER }}) (push) Has been cancelled
Build and Publish Docker Images / setup (push) Has been cancelled
This commit is contained in:
commit
68f6e85c78
17 changed files with 1286 additions and 0 deletions
193
README.md
Normal file
193
README.md
Normal file
|
@ -0,0 +1,193 @@
|
|||
# auto-transcoder
|
||||
|
||||
**Auto-transcoder** is a Python-based media transcoding application that automatically processes media files when they are added to a specified directory. It leverages Redis for tracking media state and Celery for handling transcoding tasks asynchronously. The application is containerised using Docker for easy deployment and management.
|
||||
|
||||
---
|
||||
|
||||
## 📦 Features
|
||||
|
||||
- **Automatic Transcoding**: Detects new media files in a watched directory and triggers transcoding tasks.
|
||||
- **Redis Integration**: Tracks media state (e.g., whether a file has been transcoded) using Redis.
|
||||
- **Celery Worker Support**: Uses Celery to handle transcoding tasks in the background.
|
||||
- **Docker-Ready**: Comes with Dockerfiles for both the server and worker components.
|
||||
- **Cross-Platform Compatibility**: Runs on Linux (ARM64 and AMD64 architectures).
|
||||
|
||||
---
|
||||
|
||||
## 📦 Requirements
|
||||
|
||||
- **Redis Server**: Required for media state tracking.
|
||||
- **Celery Worker**: Needed to process transcoding tasks.
|
||||
- **Docker**: For containerisation and deployment.
|
||||
|
||||
---
|
||||
|
||||
## 📁 Project Structure
|
||||
|
||||
Here are a few key files in the project structure:
|
||||
|
||||
```
|
||||
auto-transcoder/
|
||||
├── server.Dockerfile
|
||||
├── worker.Dockerfile
|
||||
├── pyproject.toml
|
||||
├── src/
|
||||
│ └── auto_transcoder/
|
||||
│ ├── model.py
|
||||
│ ├── tasks.py
|
||||
│ └── server.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
### 1. **Build Docker Images**
|
||||
|
||||
Run the following commands to build the server and worker Docker images:
|
||||
|
||||
```bash
|
||||
docker build -t auto-transcoder-server -f server.Dockerfile .
|
||||
docker build -t auto-transcoder-worker -f worker.Dockerfile .
|
||||
```
|
||||
|
||||
### 2. **Run Redis Server**
|
||||
|
||||
Ensure a Redis server is running. You can use Docker:
|
||||
|
||||
```bash
|
||||
docker run --name redis -d -p 6379:6379 redis
|
||||
```
|
||||
|
||||
### 3. **Run the Server and Worker**
|
||||
|
||||
To run the server and worker, use Docker Compose or individual commands. Here’s an example using Docker Compose:
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
redis:
|
||||
image: redis
|
||||
ports:
|
||||
- "6379:6379"
|
||||
server:
|
||||
image: auto-transcoder-server
|
||||
command:
|
||||
- /data
|
||||
- --recycle-bin
|
||||
- /data/recycle_bin
|
||||
ports:
|
||||
- "5000:5000"
|
||||
volumes:
|
||||
- /path/to/media:/data
|
||||
depends_on:
|
||||
- redis
|
||||
environment:
|
||||
- REDIS_URL=redis://redis:6379
|
||||
worker:
|
||||
image: auto-transcoder-worker
|
||||
volumes:
|
||||
- /path/to/media:/data
|
||||
depends_on:
|
||||
- redis
|
||||
environment:
|
||||
- REDIS_URL=redis://redis:6379
|
||||
```
|
||||
|
||||
Start services with:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
> 🔍 Note: Replace `/path/to/media` with the actual path to the directory you want to watch.
|
||||
|
||||
You may find a standalone docker-compose.yml example if you need help getting started [here](docker-compose.example.yml).
|
||||
|
||||
---
|
||||
|
||||
## 📌 Configuration
|
||||
|
||||
### Command Line Arguments
|
||||
|
||||
- `directory_to_watch`: Path to the directory where media files are stored. This is a required argument.
|
||||
- `--recycle-bin`: Path to a directory where original media files will be moved after transcoding. If not provided, the original files will be deleted.
|
||||
|
||||
### Environment Variables
|
||||
|
||||
- `REDIS_URL`: URL to your Redis instance (e.g., `redis://redis:6379`).
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Usage
|
||||
|
||||
1. Place media files in the watched directory.
|
||||
2. The server will automatically detect new files and add them to the queue.
|
||||
3. A Celery worker will process the transcoding tasks in the background.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 📦 Dependencies
|
||||
|
||||
This project uses the following dependencies (managed via Poetry):
|
||||
|
||||
- `celery[redis]`
|
||||
- `flask`
|
||||
- `python-magic`
|
||||
- `redis`
|
||||
- `asyncio`
|
||||
|
||||
Install them with:
|
||||
|
||||
```bash
|
||||
poetry install
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧱 Development
|
||||
|
||||
To set up the development environment:
|
||||
|
||||
1. Install [Poetry](https://python-poetry.org/docs/).
|
||||
2. Run `poetry install` to install dependencies.
|
||||
3. Run the server with `poetry run python src/auto_transcoder/server.py` (ensure Redis is running).
|
||||
4. Run the worker with `poetry run celery -A auto_transcoder.tasks worker --loglevel=info`.
|
||||
|
||||
---
|
||||
|
||||
## 📝 Notes
|
||||
|
||||
- Ensure that both the worker and server instances have access to the media files at the same path.
|
||||
- If you're using Docker, this can be achieved by mounting the media directory as a volume in your Docker Compose setup.
|
||||
- The `pyproject.toml` defines package structure and build settings. It currently uses `poetry-core` for building the package.
|
||||
- The `MediaDTO` class encapsulates media file metadata, and the `RedisManager` class is used to manage Redis connections.
|
||||
|
||||
---
|
||||
|
||||
## 📄 License
|
||||
|
||||
This project is licensed under the **MIT License**. See the [LICENCE](LICENCE) file for details.
|
||||
|
||||
---
|
||||
|
||||
## 📈 Contributing
|
||||
|
||||
Contributions are welcome! To contribute:
|
||||
|
||||
1. Fork the repository.
|
||||
2. Create a feature branch.
|
||||
3. Make your changes.
|
||||
4. Submit a pull request.
|
||||
|
||||
This repository uses British English spelling. Please ensure that your contributions adhere to this standard.
|
||||
|
||||
---
|
||||
|
||||
## 📚 References
|
||||
|
||||
- [Poetry Documentation](https://python-poetry.org/docs/)
|
||||
- [Docker Documentation](https://docs.docker.com/)
|
||||
- [Redis Documentation](https://redis.io/docs/)
|
||||
- [Celery Documentation](https://docs.celeryproject.org/)
|
Loading…
Add table
Add a link
Reference in a new issue