Commit graph

39 commits

Author SHA1 Message Date
8a882c80f7
fix(cicd): comment out Terraform Plan upload in CI/CD workflow.
All checks were successful
Terraform CI/CD / Terraform Format, Validate, Lint, Scan (push) Successful in 2m23s
Terraform CI/CD / Terraform Plan (push) Has been skipped
Terraform CI/CD / Terraform Apply (push) Successful in 3m2s
2025-06-02 17:42:28 +01:00
87b728e3c8
fix(cicd): remove trivy
All checks were successful
Terraform CI/CD / Terraform Format, Validate, Lint, Scan (push) Successful in 4m42s
Terraform CI/CD / Terraform Plan (push) Has been skipped
Terraform CI/CD / Terraform Apply (push) Successful in 3m40s
- doesn't look like trivy supports forgejo
2025-04-20 10:56:24 +00:00
85caf24a93
fix(cicd): replace tfsec with trivy
Some checks failed
Terraform CI/CD / Terraform Format, Validate, Lint, Scan (push) Failing after 7m1s
Terraform CI/CD / Terraform Plan (push) Has been skipped
Terraform CI/CD / Terraform Apply (push) Has been skipped
- tfsec is deprecated
2025-04-20 10:35:23 +00:00
ba83c03b6d
fix(cicd): create tf plugin cache so tf format check can see it
Some checks failed
Terraform CI/CD / Terraform Format, Validate, Lint, Scan (push) Failing after 6m9s
Terraform CI/CD / Terraform Plan (push) Has been skipped
Terraform CI/CD / Terraform Apply (push) Has been skipped
2025-04-20 10:21:34 +00:00
c9cbfc7979
chore(cicd): complete refactor of CICD
Some checks failed
Terraform CI/CD / Terraform Format, Validate, Lint, Scan (push) Failing after 5m27s
Terraform CI/CD / Terraform Plan (push) Has been skipped
Terraform CI/CD / Terraform Apply (push) Has been skipped
- add tfsec and tflint
2025-04-20 10:11:32 +00:00
ben
09b16119e7 chore(cicd): remove useless outcome check
All checks were successful
Lint / lint (push) Successful in 3m35s
Terraform Plan and Apply / cicd (push) Successful in 40s
2025-02-26 13:21:49 +00:00
ben
c498d6c1f2 revert cda43e2d40
All checks were successful
Lint / lint (push) Successful in 39s
Terraform Plan and Apply / cicd (push) Successful in 1m14s
revert chore(workflows): switch to ubuntu-latest-slim for lint and cicd jobs
2025-02-24 18:04:48 +00:00
cda43e2d40
chore(workflows): switch to ubuntu-latest-slim for lint and cicd jobs
Some checks failed
Lint / lint (push) Failing after 1m12s
Terraform Plan and Apply / cicd (push) Has been cancelled
2025-02-24 18:02:08 +00:00
9af9846216
feat(matrix): add Nomad job specification for Matrix service
feat(matrix): update resource allocation in Nomad job specification

feat(matrix): onboard element service to traefik

feat(matrix): add port configuration for Element service

chore(matrix): reformat

feat(matrix): update resource allocation in Nomad job specification

fix(matrix): minimum MemoryMB value is 10

feat(matrix): update resource allocation in Nomad job specification

feat(matrix): split server and clients into seperate groups

feat(matrix): well known to be served by nginx

fix(matrix): add well known route for all hosts

feat(matrix): use separate traefik router for well known

feat(matrix): migrate config.yaml for mas

feat(matrix): divide mas config between nomad and volume

feat(matrix): split cinny and element task groups

refactor(media-centre): Migrate media-centre job spec to Nomad HCL format

fix(media-centre): remove json from resource nomad job

fix(media-centre): update media-centre job spec to use Nomad HCL format

feat(media-centre): add downloader group

- Added a new group called "downloaders" to handle proxy tasks for downloading media files.
- Configured the proxy task with necessary settings and environment variables.

fix(media-centre): use OPENVPN_USER env variable in proxy task

fix(media-centre): Add /dev/net/tun device to proxy task

feat(media-centre): Add resource limits to proxy task

feat(media-centre): Add Plex task to media-centre job spec

fix(media-centre): add constraints to media-centre job spec

fix(media-centre): nomad doesn't allow sharing devices

fix(media-centre): disable change config dir ownership

fix(media-centre): plex process user is set using env vars

fix(media-centre): update PLEX_GID in job spec

fix(media-centre): update PLEX_GID in job spec

fix(media-centre): update PLEX_UID in job spec

feat(media-centre): enable nvidia gpu capabilities

feat(media-centre): add Tautulli service to media-centre job spec

fix(media-centre): update tautulli volumes

feat(plextraktsync): add plextraktsync module

fix(plextraktsync): update plextraktsync job spec "type" to "batch"

feat(plextraktsync): update resource allocation

fix(plextraktsync): fix cron schedule in plextraktsync job spec

feat(nfs-csi): add nfs-csi module

chore: update .gitignore to include .env file

chore: format files

feat(seedbox): add seedbox module

feat(seedbox): add qbittorrent module and NFS volume

feat(seedbox): add timezone configuration for seedbox job

fix(seedbox): vuetorrent-lsio-mod image env var

feat(seedbox): add HTTP_PORT environment variable for qbittorrent module

feat(seedbox): update access mode for NFS volume

feat(seedbox): add node constraint for seedbox job

feat(seedbox): add subdirectories for NFS volumes

feat(seedbox): add nolock mount flag for NFS volumes

feat(seedbox): Update NFS volume configuration

feat(seedbox): update Docker image and enable force pull

feat(seedbox): pause container network definition

feat(elk): create kibana

feat(elk): update kibana cpu allocation

feat(elk): add elasticsearch container to elk job

This commit adds a new task "elasticsearch" to the "elk" job in the "node" group. The task uses the "podman" driver and pulls the "docker.elastic.co/elasticsearch/elasticsearch:8.15.2" image with force pull enabled. It exposes the "transport" port and mounts the "/mnt/docker/elastic/elasticsearch/config" and "/mnt/docker/elastic/elasticsearch/data" volumes. The task is allocated with 500 CPU and 1024 memory resources.

feat(seedbox): update resource allocation in seedbox job

fix(elk): remove ulimit from elk job

See: https://github.com/hashicorp/nomad-driver-podman/issues/341

fix(elk): add selinuxlabel to volume mounts in elk job

refactor(modules): remove unused modules and jobspecs

refactor(elk): update CPU allocation in elk job

feat(media-centre): Plex to use host network

feat(elk): add 9200 port to es node

feat(elk): allocate more ram to node

feat(elk): allocate even more ram to node

feat(media-centre): reduce memory allocation of tautulli

feat(elk): revert memory allocation after shard tidy-up

feat(media-centre): set memory soft limit

feat(media-centre): update memory hard limit for tautulli

feat(elk): tweak node mem alloc

See: https://www.elastic.co/guide/en/elasticsearch/reference/current/size-your-shards.html#_example_11

feat(seedbox): add memory soft limit to vpn client

feat(seedbox): update memory hard limit for vpn client

fix(matrix): increase whatsapp-bridge memory allocation

refactor(elk): update elastic and kibana image versions in elk job

feat: add latest image versions and add force pull

feat: enable force pull for all podman driver tasks

feat(matrix): increase syncv3 memory allocation

feat: migrate podman memory allocation to nomad max memory

fix: nomad max memory is defined by memory_max

feat(matrix): add ecs fields to task metadata

refactor(matrix): migrate shared meta to parent

refactor(matrix): update resource allocation in jobspec.nomad.hcl

refactor(matrix): update resource allocation in jobspec.nomad.hcl

refactor(matrix): update resource allocation in jobspec.nomad.hcl

refactor(plextraktsync): update resource allocation in jobspec.nomad.hcl

refactor(plextraktsync): remove task node constraint

refactor: migrate podman tasks to docker tasks

feat(elk): update ulimit for elasticsearch container

refactor(elk): update volume paths in jobspec.nomad.hcl

feat(seedbox): remove pause container

feat(elk): update kibana count in jobspec.nomad.hcl

refactor(elk): remove node constraint from kibana

refactor(elk): add spread attribute to kibana

refactor(elk): update port configuration in jobspec.nomad.hcl

refactor(dummy): migrate json jobspec to hcl

feat(dummy): update service provider to consul

fix(dummy): add port label to port definition

refactor(dummy): rename jobspec to match standard

feat(dummy): migrate to service mesh

chore(dummy): update Nomad provider version to 2.4.0

chore(dummy): update Nomad provider version to 2.4.0

feat(dummy): configure traefik

refactor(dummy): update provider to use consul instead of nomad

feat(renovate): create module for automated dependency updates

Add renovate.json

fix(renovate): increase memory allocation

feat(renovate): add GITHUB_COM_TOKEN variable

refactor(renovate): pin version

feat(renovate): enable dependency dashboard

refactor(matrix): use bridge netowrking for server group

refactor(matrix): update URLs to use allocated addresses

refactor(matrix): remove host.docker.internal host

fix(matrix): update SYNCV3_BINDADDR

fix(matrix): update SYNCV3_BINDADDR port to 8009

fix(elk): increase memory allocation

feat(elk): disable co-located kibana allocations

refactor(jobspec): update provider to consul for elk and media-centre services

feat(media-centre): reduce memory allocation from 4096 to 1024

fix(jobspec): replace constraints with new neto client id

feat(elk): update data volume path to use unique node name

feat(elk): migrate elastic config to nfs

feat(elk): add Nyx

refactor(workflows): reformat (#17)

Reviewed-on: #17

fix(elk): increase memory allocation to 2048 MB

refactor(matrix): remove specific node constraint from job specification

feat(matrix): implement consul service mesh

feat(elk): use allocation index for node state location

refactor(media-centre): remove deprecated NVIDIA_DRIVER_CAPABILITIES

fix(media-centre): plex transcode dir not writable

fix(media-centre): set transcode dir to world writable

fix(media-centre): set transcode dir to world writable

feat(media-centre): replace plex transcode dir with a persistent volume

feat(media-centre): increase plex memory limit

For caching

chore(elk): promote elastic version

feat(elk): remove force_pull option from Elasticsearch and Kibana configurations

style(jobspec): improve formatting in HCL files

feat(elk): add health check

feat(media-centre): add NVIDIA visible devices for Jellyfin and Plex

fix(media-centre): increase max memory for tautulli

feat(plugin-csi): add NFS CSI driver jobspec and main configuration

feat(main.tf): add plugin-csi module to main configuration

fix(plugin-csi): refactor NFS job specifications into separate files for controller and node

fix(plugin-csi): add NFS path variable for controller and node resources

fix(plugin-csi): add NFS path variable to controller and node job specifications

fix(plugin-csi): add provisioner name to NFS job specifications for controller and node

fix(plugin-csi): update NFS job specifications

feat(seedbox): restructure job specifications and add NFS volume registrations for media and qbittorrent config

feat(workflows): add lint workflow for Terraform and Nomad formatting

fix(seedbox): add attachment and access modes for media and qbittorrent_config volumes

feat(seedbox): remove node constraint

Update modules/seedbox/main.tf

fix(seedbox): add mount options with nolock flag for media and qbittorrent_config volumes

fix(seedbox): update share paths to use lowercase in media and qbittorrent_config volumes

fix(seedbox): remove unused device configuration from jobspec

feat(matrix): add health check configuration

feat(matrix): add health check ports for synapse, mas, and nginx

fix(matrix): remove health check configuration for synapse, mas, and nginx

feat(main.tf): remove unused and broken seedbox module

feat(renovate): use JSON log format

chore(elk): upgrade version to latest

feat(elk): use 2 kibana replicas

feat(elk): add on_update ignore option to ready check configuration

fix(elk): update volume paths to use node unique name for configuration and data

feat(matrix): add envoy_metrics port and update service metadata for Consul integration

feat(matrix): add health check configuration to synapse job

feat(matrix): add /metrics endpoint exposure for envoy_metrics

fix(matrix): update service port configurations to use static port numbers

feat(matrix): restructure ingress groups and enhance service configurations for improved routing

fix(matrix): update whatsapp bridge tokens and change push to receive ephemeral

feat(media-centre): remove node constraint from tautulli task configuration

feat(elk): onboard hestia node to nomad

feat(elk): enhance job specification with Envoy metrics and update service configurations

feat(renovate): onboard nomad docker image updates

chore(deps): update ghcr.io/renovatebot/renovate docker tag to v38.142.7

chore(jobspec): use explicit image version tags where possible

fix(jobspec): formatting

chore(deps): update busybox docker tag to v1.37.0

chore(deps): update docker.io/library/nginx docker tag to v1.27.3

chore(deps): update ghcr.io/renovatebot/renovate docker tag to v39

chore(deps): update ghcr.io/renovatebot/renovate docker tag to v39.59.0

chore(deps): update ghcr.io/renovatebot/renovate docker tag to v39.60.0

chore(matrix): format multiline string in jobspec.nomad.hcl for improved readability

chore(secrets): refactor jobspecs to use templates for sensitive environment variables
2024-12-17 13:56:26 +00:00
cd88bac77d
feat: Remove Nomad validation step in plan-and-apply workflow 2024-08-04 12:40:18 +00:00
05d7a87306 Resolve configuration issues (#8)
Reviewed-on: #8
2024-06-15 15:59:43 +01:00
70aff7bed1 Create Jellyfin job (#6)
Closes #5

Reviewed-on: #6
2024-06-15 00:22:02 +01:00
cf3b9f4518 Add NOMAD_TOKEN to CICD env vars 2024-06-09 11:19:40 +01:00
c738d2e9fc
Do not distribute tfplan
tfplan is found to store tfstate
2024-06-06 22:15:15 +01:00
53377dccc6
Call terraform init in deployment job to recreate link to plugin 2024-06-06 22:06:47 +01:00
407301bd92
No longer apply permissions changes to plugins 2024-06-06 22:01:52 +01:00
ben
ba5f3704d4 Use caching instead of artifacts to store .terraform (#2)
Co-authored-by: Ben Martin <ben.martin@sky.uk>
Reviewed-on: #2
2024-06-06 22:00:05 +01:00
6b2f1c473f
Inject PG_CONN_STR secret into env vars in CICD 2024-06-06 20:43:55 +01:00
73e06ac643
Ensure providers are executable 2024-06-04 01:32:59 +01:00
81e94931ed
Package whole .terraform 2024-06-04 01:27:37 +01:00
00c4161fa2
Tree isn't available in bin 2024-06-04 01:25:57 +01:00
e5031ad8cb
Download artifact doesn't nest 2024-06-04 01:25:00 +01:00
04cbd49f8c
Debugging 2024-06-04 01:23:31 +01:00
f4e1125894
Artifact names can not have slashes 2024-06-04 01:21:40 +01:00
adcc006d74
Trial nesting issues 2024-06-04 01:20:42 +01:00
bfc0883976
Explicitly set artifact download path 2024-06-04 01:17:01 +01:00
ac7b147b6c
Use filename as artifact name 2024-06-04 01:06:49 +01:00
4614e07aee
Persist .terraform 2024-06-04 01:01:19 +01:00
3abf8019f5
Install nomad for deploy 2024-06-04 00:56:53 +01:00
e5b904b9e0
Download artifact to workspace 2024-06-04 00:49:10 +01:00
230ef022ce
Upload artifact V4 doesn't appear to be supported 2024-06-04 00:36:26 +01:00
539339fb06
Upload artifact V4 doesn't appear to be supported 2024-06-04 00:34:25 +01:00
56037fc11e
Use artifacts to persist tfplan 2024-06-04 00:28:39 +01:00
68fc0dc1e0
Merge apply with plan 2024-06-04 00:25:26 +01:00
0a1960a891
HOTFIX: correct deployment workflow dependency 2024-06-04 00:06:27 +01:00
065b5996b3
HOTFIX: correct deployment workflow dependency 2024-06-04 00:05:35 +01:00
dc293304ea
HOTFIX: rename job name 2024-06-04 00:03:25 +01:00
cba0f1cfd4 Integrate with actions (#1)
Co-authored-by: Ben Martin <ben.martin@sky.uk>
Reviewed-on: #1
2024-06-04 00:00:20 +01:00
d4335ca10e
Create gitea workflow 2024-06-03 21:42:32 +01:00