Compare commits

...

78 Commits

Author SHA1 Message Date
claude code agent
0bf4e71b79 fix(stacks): keep the stack breadcrumb trail on container attribute sub-tabs
When a container is opened from a stack, the detail tab kept the stack
trail (PR #7) but the attribute sub-tabs (Logs, Stats, Inspect, Console,
Attach) dropped it: those tabs were registered only under the global
docker.containers.container.* tree, so navigating to one left the stack
state (and its inherited params) behind, and each sub-view set a hardcoded
"Containers > ..." breadcrumb.

- Register stack-scoped child states docker.stacks.stack.container.{attach,
  exec,inspect,logs,stats} mirroring the global ones, so the inherited stack
  params survive and the trail can be kept.
- Centralize the breadcrumb logic in containerBreadcrumbs.ts (moved out of
  ItemView, which re-exports it) and add isStackContainerState +
  getContainerSubTabBreadcrumbs + buildStackContainerLinkParams.
- ActionLinksRow links sub-tabs into the stack tree (with stack+container
  params) when opened from a stack, else the global states unchanged.
- InspectView + the logs/stats/console controllers render the stack-aware
  trail; set up-front (no name) so it survives the load window and errors.

Covers regular/external/orphaned stacks and the non-stack fallback,
matching the existing ItemView breadcrumb behavior. New unit tests in
containerBreadcrumbs.test.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 02:25:04 +03:00
claude code agent
e9fae32b43 test(stacks): cover orphaned branch; name the stack-container state; type link params (F1,F2,F4)
Maintainer pre-merge review follow-up:
F1: test the orphaned-stack breadcrumb branch (orphaned=true, no regular) —
    href carries stackId/orphaned, not external.
F2: extract STACK_CONTAINER_STATE_NAME so code + test share one literal.
F4: type buildStackLinkParams' return as StackLinkParams (documents the real
    shape; external stays boolean, serialized by ui-router — no runtime change).
F3 (legacy ?id= deep links) answered wontfix in the PR thread.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:55:41 +03:00
claude code agent
a1851417d1 test(stacks): cover external-stack breadcrumb branch in buildStackLinkParams (F1)
Add a test case driving the external-stack branch (external='true', no DB
stackId) and assert the back-link carries external=true/type and omits
stackId/regular. stackId/regular are set in the route params so the negative
assertions actually catch a fall-through-to-regular regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:06:25 +03:00
claude code agent
b4d10a67b2 fix(stacks): preserve stack tab on breadcrumb back-link + assert default crumb (F1,F2)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 07:55:47 +03:00
claude code agent
cb11b0fca4 fix(stacks): keep stack breadcrumb trail when opening a container from a stack (#4)
Opening a container from a stack's Containers table showed
"Home > Containers > <container>" instead of keeping the stack trail,
so the user could not navigate back to the stack.

Two root causes are addressed:

1. Route param collision: docker.stacks.stack used the query param `id`
   for the numeric stack DB id, while its child docker.stacks.stack.container
   uses the path param `id` for the container id. Navigating into a container
   overwrote the stack id. The stack id param is renamed `id` -> `stackId`
   everywhere it is read or written (route url, stacks datatable link,
   create-stack redirect, gitops workflow card link, stack ItemView reader).

2. Hardcoded breadcrumbs: the container details ItemView always rendered the
   global "Containers" crumb. Breadcrumbs are now state-aware: when reached
   via docker.stacks.stack.container the stack trail
   (Stacks > <stack> > <container>) is rebuilt from the inherited stack params,
   honoring external/orphaned stacks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 07:44:18 +03:00
vvzvlad
9e9bb1bbff ci: install client deps with --no-frozen-lockfile explicitly
Some checks failed
Build image / build (push) Has been cancelled
pnpm ignores the npm_config_frozen_lockfile env var, so the previous fix did not take effect and CI still ran a frozen install. Add an explicit 'pnpm install --no-frozen-lockfile' step before 'make build-all' to reconcile the lockfile (missing pnpmfileChecksum for configDependencies); the subsequent frozen install in 'make client-deps' then succeeds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 19:21:11 +03:00
vvzvlad
d0a0395337 ci: disable pnpm frozen-lockfile in build workflow
CI enables --frozen-lockfile by default, which fails with ERR_PNPM_LOCKFILE_CONFIG_MISMATCH because the committed pnpm-lock.yaml does not record the pnpmfileChecksum for the configDependencies declared in package.json. Set npm_config_frozen_lockfile=false so the bare 'pnpm install' run by 'make client-deps' reconciles the lockfile instead of failing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 19:18:59 +03:00
vvzvlad
88589e4cb3 ci: add GitHub Actions workflow to build and publish fork image to GHCR
Builds the CE fork (frontend + Go server) and publishes a drop-in compatible image to ghcr.io/vvzvlad/portainer-ce on pushes to develop, v* tags, and manual dispatch. Single-arch linux/amd64, alpine base, production frontend build (ENV=production), tags <package.json version> and latest, GHCR auth via the built-in GITHUB_TOKEN.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 19:14:36 +03:00
vvzvlad
af74986e66 chore: add VS Code workspace file 2026-06-28 18:55:18 +03:00
andres-portainer
e664bf0e19 fix(helm): add missing SSRF protections BE-13136 (#3001) 2026-06-22 20:25:10 -03:00
nickl-portainer
152c89972b chore(eslint): update eslint to latest v9 [R8S-1090] (#2954) 2026-06-23 11:04:33 +12:00
Oscar Zhou
25c69c6e9b fix(ui): update server installation timeout redirect link [BE-13124] (#2991) 2026-06-23 08:49:43 +12:00
andres-portainer
a6370808ae fix(ssrf): disable HTTP/2 for some specific cases BE-13121 (#2996) 2026-06-22 16:13:43 -03:00
Chaim Lev-Ari
6bfd2360d8 docs(security): add FAQ link to setup token messages [BE-13125] (#2995)
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-22 20:30:49 +03:00
Chaim Lev-Ari
872d1e03f6 feat(gitops): add "create new source" button to GitSourceSelector [BE-13054] (#2960)
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-22 17:19:53 +03:00
Chaim Lev-Ari
a5cacd712d refactor(gitops): remove manual credential entry from git form [BE-13047] (#2951)
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-22 15:42:09 +03:00
Phil Calder
f596c862b3 fix(websocket): enforce environment authorization on kubernetes-shell [BE-13027] (#2774)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: oscarzhou <oscar.zhou@portainer.io>
2026-06-22 15:09:41 +12:00
bernard-portainer
5395dee4c6 feat(gpu-stats): add gpu stats to environments [C9S-200] (#2735) 2026-06-22 09:21:43 +12:00
Josiah Clumont
217fe870ef fix(git): use ListContext instead of List when fetching remote refs [C9S-263] (#2939) 2026-06-22 08:30:20 +12:00
andres-portainer
26334e9088 feat(ssrf): add missing transport wrappings and more checks BE-13021 (#2968) 2026-06-19 20:26:03 -03:00
RHCowan
cc45af2873 fix(lint): enforce consistent golangci-lint version across CI and pre-commit [PLA-777] (#2966) 2026-06-19 11:45:12 +12:00
RHCowan
37bd8c06b5 fix(security): gate docker dashboard and edge async command routes [R8S-1057] (#2953) 2026-06-19 11:08:01 +12:00
andres-portainer
c821a1c59f fix(git): avoid cloning to memory and bypassing symlinking restriction BE-13115 (#2961) 2026-06-18 16:21:09 -03:00
Dakota Walsh
f5d0b3d849 feat(kubernetes): Gateway api client included in kubeclient [C9S-244] (#2884) 2026-06-18 14:37:42 +12:00
nickl-portainer
0dfd27f08c fix(pnpm): pnpm format command failing [R8S-1071] (#2932) 2026-06-18 13:27:01 +12:00
nickl-portainer
0dfa0266c7 fix(webpack): update shell-quote [R8S-1074] (#2934) 2026-06-17 10:50:48 +12:00
nickl-portainer
9b807ca314 fix(axios): update axios [R8S-1075] (#2935) 2026-06-17 10:50:34 +12:00
nickl-portainer
de5d84ade4 fix(kubernetes): handling undefined responseStatus [R8S-1072] (#2933) 2026-06-17 09:32:59 +12:00
Chaim Lev-Ari
4d539a691d feat(custom-templates): reuse existing git sources in create/update [BE-13053] (#2925)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 21:45:35 +03:00
Chaim Lev-Ari
ee8e73d7f9 feat(edge/stacks): use source ID for edge stack git creation [BE-13044] (#2926)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 17:33:19 +03:00
Chaim Lev-Ari
32c6bedb98 feat(stacks): use source for kubernetes manifest git stacks [BE-13045] (#2915)
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-16 14:35:16 +03:00
Ali
cd9bb18ba1 feat(policies): reuse filter status component, give consistent styles [c9s-210] (#2723) 2026-06-16 15:58:33 +12:00
nickl-portainer
f365035563 fix(git): update lint-staged to v17 [R8S-1071] (#2907) 2026-06-16 15:14:57 +12:00
Chaim Lev-Ari
d9673e33ec feat(helm): reuse existing git sources in Kubernetes Helm-from-git install [BE-13046] (#2900)
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-15 22:01:31 +03:00
Chaim Lev-Ari
491df61fbf chore(hey-api): disable api validator [BE-13102] (#2918) 2026-06-15 21:35:19 +03:00
Chaim Lev-Ari
ca1d9dc6a2 fix(edge/stacks): load envs by id [BE-13097] (#2917) 2026-06-15 21:23:49 +03:00
andres-portainer
16b5554f66 fix(customtemplates): add resource controls BE-13019 (#2897) 2026-06-15 14:59:07 -03:00
Chaim Lev-Ari
fcdd6b4510 feat(stacks): use source id to create git stacks [BE-13043] (#2870)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 18:49:26 +03:00
Chaim Lev-Ari
04048c3818 fix(api): update environment status field to be optional [BE-13070] (#2847) 2026-06-15 13:36:12 +03:00
Chaim Lev-Ari
1afbc621a4 fix(editor): restore yaml syntax highlighting in web editor [BE-13073] (#2848)
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-15 10:50:02 +03:00
Devon Steenberg
ef807950f1 fix(compose-unpacker): port swarm commands to use libstack [BE-12915] (#2890) 2026-06-15 11:43:29 +12:00
Phil Calder
d37f3aa504 chore(server-ce): update Code of Conduct contact to contribute@portainer.io (#2889)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:44:41 +12:00
Oscar Zhou
39b3eb3d64 fix(registry): standard user with access permission cannot browse and delete private images [BE-13072] (#2877) 2026-06-13 19:52:03 +12:00
Devon Steenberg
8b21dfc318 feat(ssrf): add ssrf allow list to settings [BE-13021] (#2858) 2026-06-12 15:16:06 +12:00
Steven Kang
f87fec6d61 fix(omni): prevent partial cluster creation on Talos/Kubernetes mismatch [R8S-1058] (#2849)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 11:50:16 +12:00
Ali
391eb22d98 fix(ui): ui consistency and bug fixes [r8s-1061] (#2880) 2026-06-12 11:49:45 +12:00
andres-portainer
0da42c01b6 feat(gitcredential): remove GitCredential BE-12919 (#2838) 2026-06-11 18:53:24 -03:00
Cara Ryan
f3f0ca8e21 fix(rbac): Filter get namespace by allowed namespace list [SEC-61] (#2743) 2026-06-11 15:51:32 +12:00
RHCowan
96dc79e253 feat(alerting): add Kubernetes API server high request latency alert [R8S-1049] (#2765)
Co-authored-by: Steven Kang <skan070@gmail.com>
2026-06-11 15:44:33 +12:00
Xing
ac3416c5a2 feat(policies): define ObservabilityK8s policy type with deploy-and-connect and connect-only modes [C9S-121] (#2706)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 15:09:33 +12:00
Cara Ryan
ade5b2a3db feat(rbac): Add toggle for additive kubernetes RBAC policy [C9S-177] (#2814) 2026-06-11 13:44:43 +12:00
Steven Kang
1cd6017df6 fix(api): add endpoint authorization check to /api/kubernetes/{id}/* route - develop [R8S-1056] (#2829) 2026-06-11 09:49:50 +12:00
Oscar Zhou
06caea7b16 fix(security): bump golang to 1.26.4 [DEV-91] (#2866) 2026-06-11 09:31:35 +12:00
Oscar Zhou
114779d3af fix(security): bump go-git/v5 to 5.19.1 [DEV-92] (#2863) 2026-06-11 09:31:07 +12:00
nickl-portainer
96d694b66b fix(storybook): add row id for example data for datatable story [R8S-1062] (#2856) 2026-06-11 09:10:29 +12:00
andres-portainer
babb4ffb37 fix(nolint): remove unnecessary nolint directives BE-13074 (#2852) 2026-06-10 15:35:08 -03:00
LP B
0c2f07988a feat(app/sources): source create view (#2680)
Co-authored-by: Chaim Lev-Ari <chaim.lev-ari@portainer.io>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 21:34:46 +03:00
Ali
d7a1d34be7 feat(policies): docker cleanup policy [c9s-87] (#2681) 2026-06-10 16:17:23 +12:00
nickl-portainer
6a465637d4 feat(components): add new FilePicker component [R8S-1050] (#2754) 2026-06-10 10:34:14 +12:00
andres-portainer
154c19403a fix(chisel): release a lock earlier to avoid a deadlock and clean stale tunnels immediately BE-13050 (#2815) 2026-06-09 10:41:05 -03:00
bernard-portainer
c9e1467244 fix(stats-items): ensure stats items have consistent widths [C92-215] (#2844) 2026-06-09 16:53:17 +12:00
andres-portainer
1765e41fd4 feat(ssrf): implement an SSRF protection mechanism BE-13021 (#2818) 2026-06-09 00:41:42 -03:00
Phil Calder
d34ee82754 docs(contributing): fix reversed markdown link syntax for swag docs [DEV-89] (#2841)
Co-authored-by: ferreiraborgesaxel-design <ferreiraborgesaxel-design@users.noreply.github.com>
2026-06-09 14:25:50 +12:00
Oscar Zhou
5cdd0023d7 fix(registry): suppress ecr token pre-validation error with warning log [BE-13059] (#2827) 2026-06-09 10:58:50 +12:00
andres-portainer
df7a4b5d6f feat(gitops): improve the data model BE-12919 (#2819) 2026-06-08 15:01:55 -03:00
Chaim Lev-Ari
63eb96859d fix(endpoints): parse endpoint on GET [BE-13049] (#2824) 2026-06-08 16:59:09 +03:00
Josiah Clumont
e3e2a3b782 fix(environments): Environment Groups detail view environment breakdown regression [BE-13051] (#2828) 2026-06-08 16:03:32 +12:00
Steven Kang
eeafa5e0a5 fix(security): bump containerd to 1.7.32 and containerd/v2 to 2.2.4 - develop [DEV-79] (#2812) 2026-06-08 11:09:34 +12:00
Chaim Lev-Ari
7e5e71ae67 chore(deps): bump patch/minor frontend dependencies [BE-13004] (#2808)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 15:45:58 +03:00
andres-portainer
8daf0bb2a9 feat(customtemplates): use Sources for CustomTemplates BE-12919 (#2759) 2026-06-05 01:51:18 -03:00
nickl-portainer
a779c839b7 feat(kubernetes): set application list default to expanded [R8S-1040] (#2758) 2026-06-05 14:35:00 +12:00
Josiah Clumont
0da57f8747 fix(environment-groups): restore dark mode badge colors and fix empty list regression [C9S-231] (#2811) 2026-06-05 11:58:54 +12:00
Steven Kang
d01d241af1 fix(security): bump golang.org/x/net to v0.55.0 - develop [DEV-77] (#2797) 2026-06-05 10:46:41 +12:00
Steven Kang
dd08d09d14 fix(security): bump Go to 1.26.3 to remediate 8 stdlib CVEs - develop [DEV-78] (#2800) 2026-06-05 10:46:22 +12:00
Ali
0143393a8c chore(policies): use generic policy reconcile system so more than helm can be used [c9s-88] (#2613) 2026-06-05 07:47:25 +12:00
Chaim Lev-Ari
d2b56efcb4 feat(security): require setup token for admin init and restore [BE-13029] (#2770) 2026-06-04 09:15:23 +03:00
Chaim Lev-Ari
dab0cf48c6 docs(api): document CE type generation [BE-13031] (#2794)
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-04 09:13:45 +03:00
Hannah Cooper
916367dccb fix(api-docs): time.Duration bounds fix + linting fixes [C9S-223] (#2762) 2026-06-04 15:14:07 +12:00
446 changed files with 17715 additions and 12313 deletions

View File

@@ -3,13 +3,13 @@ body:
attributes:
value: |
# Welcome!
Thanks for suggesting an idea for Portainer!
Before opening a new idea or feature request, make sure that we do not have any duplicates already open. You can ensure this by [searching this discussion category](https://github.com/orgs/portainer/discussions/categories/ideas). If there is a duplicate, please add a comment to the existing idea instead.
Also, be sure to check our [knowledge base](https://portal.portainer.io/knowledge) and [documentation](https://docs.portainer.io) as they may point you toward a solution.
**DO NOT FILE DUPLICATE REQUESTS.**
- type: textarea

86
.github/workflows/build-image.yml vendored Normal file
View File

@@ -0,0 +1,86 @@
name: Build image
on:
push:
branches: [develop]
tags: ['v*']
workflow_dispatch: {}
env:
IMAGE: ghcr.io/vvzvlad/portainer-ce
jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup pnpm
uses: pnpm/action-setup@v4
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 22
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
- name: Resolve version
id: ver
run: echo "version=$(node -p "require('./package.json').version")" >> "$GITHUB_OUTPUT"
- name: Install client dependencies
# CI forces pnpm into --frozen-lockfile, which fails with
# ERR_PNPM_LOCKFILE_CONFIG_MISMATCH because the committed lockfile lacks
# the pnpmfileChecksum for the configDependencies in package.json.
# Reconcile the lockfile explicitly; the later frozen install in
# `make client-deps` then finds a matching lockfile. pnpm ignores the
# npm_config_frozen_lockfile env var, so an explicit flag is required.
run: pnpm install --no-frozen-lockfile
- name: Build client and server
env:
SKIP_GO_GET: "true"
CONTAINER_IMAGE_TAG: ${{ steps.ver.outputs.version }}
BUILDNUMBER: ${{ github.run_number }}
# Pin the embedded commit to the full SHA so it matches the image
# GIT_COMMIT build-arg and does not depend on the shallow checkout.
GIT_COMMIT_HASH: ${{ github.sha }}
# ENV=production selects webpack/webpack.production.js (minified bundle),
# matching the official CE image; the Makefile default is development.
run: make build-all ENV=production
- name: Ensure storybook directory exists
# make build-all does not produce dist/storybook, but alpine.Dockerfile
# has `COPY dist/storybook* /storybook/`; without a match the docker build fails.
run: mkdir -p dist/storybook
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push image (linux/amd64, alpine base)
uses: docker/build-push-action@v6
with:
context: .
file: build/linux/alpine.Dockerfile
platforms: linux/amd64
push: true
tags: |
${{ env.IMAGE }}:${{ steps.ver.outputs.version }}
${{ env.IMAGE }}:latest
build-args: |
GIT_COMMIT=${{ github.sha }}

View File

@@ -1,4 +1,4 @@
version: "2"
version: '2'
linters:
default: none
enable:

View File

@@ -1,10 +1,11 @@
version: "2"
version: '2'
run:
allow-parallel-runners: true
linters:
default: none
enable:
- gocritic
- bodyclose
- copyloopvar
- depguard
@@ -31,7 +32,7 @@ linters:
- exptostd
settings:
staticcheck:
checks: ["all", "-ST1003", "-ST1005", "-ST1016", "-SA1019", "-QF1003"]
checks: ['all', '-ST1003', '-ST1005', '-ST1016', '-SA1019', '-QF1003']
depguard:
rules:
main:
@@ -76,6 +77,13 @@ linters:
desc: use github.com/Masterminds/semver/v3
- pkg: github.com/hashicorp/go-version
desc: use github.com/Masterminds/semver/v3
gocritic:
disable-all: true
enabled-checks:
- ruleguard
settings:
ruleguard:
rules: './analysis/ssrf.go,./analysis/git.go'
forbidigo:
forbid:
- pattern: ^tls\.Config$
@@ -83,9 +91,11 @@ linters:
- pattern: ^tls\.Config\.(InsecureSkipVerify|MinVersion|MaxVersion|CipherSuites|CurvePreferences)$
msg: Do not set this field directly, use crypto.CreateTLSConfiguration() instead
- pattern: ^object\.(Commit|Tag)\.Verify$
msg: "Not allowed because of FIPS mode"
msg: 'Not allowed because of FIPS mode'
- pattern: ^(types\.SystemContext\.)?(DockerDaemonInsecureSkipTLSVerify|DockerInsecureSkipTLSVerify|OCIInsecureSkipTLSVerify)$
msg: "Not allowed because of FIPS mode"
msg: 'Not allowed because of FIPS mode'
- pattern: ^git\.PlainClone(Context|WithOptions)?$
msg: Use git.CloneContext with NewNoSymlinkFS to prevent symlink traversal attacks
analyze-types: true
exclusions:
generated: lax
@@ -93,6 +103,14 @@ linters:
- comments
- common-false-positives
- legacy
rules:
- path: pkg/libhttp/ssrf
linters:
- gocritic
text: ruleguard
- path: pkg/libhttp/ssrf/builder\.go
linters:
- forbidigo
paths:
- third_party$
- builtin$

View File

@@ -1,3 +1,5 @@
dist
api/datastore/test_data
coverage
coverage
pnpm-lock.yaml

View File

@@ -8,19 +8,19 @@ In the interest of fostering an open and welcoming environment, we as contributo
Examples of behavior that contributes to creating a positive environment include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
- Using welcoming and inclusive language
- Being respectful of differing viewpoints and experiences
- Gracefully accepting constructive criticism
- Focusing on what is best for the community
- Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
- The use of sexualized language or imagery and unwelcome sexual attention or advances
- Trolling, insulting/derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or electronic address, without explicit permission
- Other conduct which could reasonably be considered inappropriate in a professional setting
## Our Responsibilities
@@ -34,7 +34,7 @@ This Code of Conduct applies both within project spaces and in public spaces whe
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at anthony.lapenna@portainer.io. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at contribute@portainer.io. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.

View File

@@ -147,7 +147,9 @@ When adding a new route to an existing handler use the following as a template (
// @router /{id} [get]
```
explanation about each line can be found (here)[https://github.com/swaggo/swag#api-operation]
explanation about each line can be found [here](https://github.com/swaggo/swag#api-operation)
After changing these annotations, regenerate the TypeScript API client and types — see [Generating API types](./README.md#generating-api-types).
## Licensing

View File

@@ -6,6 +6,7 @@ TAG=local
SWAG=go run github.com/swaggo/swag/cmd/swag@v1.16.6
GOTESTSUM_VERSION?=v1.13.0
GOTESTSUM=go run gotest.tools/gotestsum@$(GOTESTSUM_VERSION)
GOLANGCI_LINT_VERSION := $(shell cat $(shell git rev-parse --show-toplevel)/.golangci-version)
# Don't change anything below this line unless you know what you're doing
.DEFAULT_GOAL := help
@@ -90,13 +91,25 @@ format-server: ## Format server code
go fmt ./...
##@ Lint
.PHONY: lint lint-client lint-server
.PHONY: lint lint-client lint-server check-lint-version
lint: lint-client lint-server ## Lint all code
lint-client: ## Lint client code
pnpm run lint
lint-server: tidy ## Lint server code
check-lint-version:
@installed=v$$(golangci-lint --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1); \
if [ "$$installed" = "v" ]; then \
echo "ERROR: golangci-lint not found, need $(GOLANGCI_LINT_VERSION)"; \
echo "Install: go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@$(GOLANGCI_LINT_VERSION)"; \
exit 1; \
elif [ "$$installed" != "$(GOLANGCI_LINT_VERSION)" ]; then \
echo "ERROR: golangci-lint $$installed installed, need $(GOLANGCI_LINT_VERSION)"; \
echo "Install: go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@$(GOLANGCI_LINT_VERSION)"; \
exit 1; \
fi
lint-server: tidy check-lint-version ## Lint server code
golangci-lint run --timeout=10m -c .golangci.yaml
golangci-lint run --timeout=10m --new-from-rev=HEAD~ -c .golangci-forward.yaml
@@ -109,7 +122,7 @@ dev-extension: build-server build-client ## Run the extension in development mod
.PHONY: docs-build docs-validate docs-clean docs-validate-clean
docs-build: init-dist ## Build docs
go mod download
cd api && $(SWAG) init -o "../dist/docs" -ot "yaml" -g ./http/handler/handler.go --parseDependency --parseInternal --parseDepth 2 -p pascalcase --markdownFiles ./
cd api && $(SWAG) init -o "../dist/docs" -ot "yaml" -g ./http/handler/handler.go --parseDependency --parseInternal --parseDepth 2 -p pascalcase --markdownFiles ./ --overridesFile .swaggo
docs-validate: docs-build ## Validate docs
pnpm swagger2openapi --warnOnly dist/docs/swagger.yaml -o dist/docs/openapi.yaml

View File

@@ -44,6 +44,32 @@ You can join the Portainer Community by visiting [https://www.portainer.io/join-
- Want to report a bug or request a feature? Please open [an issue](https://github.com/portainer/portainer/issues/new).
- Want to help us build **_portainer_**? Follow our [contribution guidelines](https://docs.portainer.io/contribute/contribute) to build it locally and make a pull request.
## Generating API types
The frontend consumes a TypeScript API client (SDK functions and request/response types) that is generated from the Go API's Swagger annotations. Regenerate it after any API change — a new endpoint, a changed request/response shape, or a removed endpoint:
```bash
make generate-api
```
This runs the following pipeline:
```
Go Swagger annotations
→ dist/docs/swagger.yaml (make docs-build, via swaggo/swag)
→ dist/docs/openapi.yaml (swagger2openapi + validation)
→ app/react/portainer/generated-api/portainer/ (hey-api/openapi-ts)
```
The generator is configured in [`openapi-ts.config.ts`](./openapi-ts.config.ts), which controls the output path, plugins, and tag filters (for example, `deprecated` endpoints and `edge_agent`-tagged routes are excluded).
The generated files live in `app/react/portainer/generated-api/portainer/` and must **not** be edited by hand — your changes would be overwritten on the next run. Import the generated SDK functions and types instead of writing direct HTTP calls:
- `@api/sdk.gen` — SDK functions
- `@api/types.gen` — request/response types
See [Adding api docs](./CONTRIBUTING.md#adding-api-docs) for how to annotate handlers so they are picked up by the generator.
## Security
For information about reporting security vulnerabilities, please see our [Security Policy](SECURITY.md).

18
analysis/git.go Normal file
View File

@@ -0,0 +1,18 @@
//go:build ignore
package gorules
import "github.com/quasilyte/go-ruleguard/dsl"
// inMemoryCloneWithWorktree flags git clone calls that use memory.NewStorage() as
// the storer while also writing files to a real worktree. This holds all git objects
// in heap for the duration of the clone, which is unbounded for user-supplied repos.
func inMemoryCloneWithWorktree(m dsl.Matcher) {
m.Match(`git.CloneContext($_, memory.NewStorage(), $wt, $_)`).
Where(m["wt"].Text != "nil").
Report(`git.CloneContext with memory.NewStorage() holds all git objects in heap; use gogitfs.NewStorage with a filesystem storer instead`)
m.Match(`git.Clone(memory.NewStorage(), $wt, $_)`).
Where(m["wt"].Text != "nil").
Report(`git.Clone with memory.NewStorage() holds all git objects in heap; use gogitfs.NewStorage with a filesystem storer instead`)
}

75
analysis/ssrf.go Normal file
View File

@@ -0,0 +1,75 @@
//go:build ignore
package gorules
import "github.com/quasilyte/go-ruleguard/dsl"
// unwrappedHTTPTransport flags any bare http.Transport composite literal.
// All transports must be created via ssrf.NewTransport or ssrf.NewInternalTransport,
// which clone http.DefaultTransport and handle SSRF protection internally.
func unwrappedHTTPTransport(m dsl.Matcher) {
m.Match(`$f(&http.Transport{$*_})`).
Report(`$f receives a bare *http.Transport; use ssrf.NewTransport(tlsConfig) or ssrf.NewInternalTransport(tlsConfig) instead`)
m.Match(`$_ := &http.Transport{$*_}`).
Report(`bare *http.Transport variable; use ssrf.NewTransport(tlsConfig) or ssrf.NewInternalTransport(tlsConfig) instead`)
m.Match(`$_.Transport = &http.Transport{$*_}`).
Report(`bare *http.Transport field assignment; use ssrf.NewTransport(tlsConfig) or ssrf.NewInternalTransport(tlsConfig) instead`)
}
// helmGetterTransport flags getter.WithTransport calls that receive a bare *http.Transport.
// Helm v4 installs its own transport and bypasses http.DefaultTransport, so the transport
// passed here must be created via ssrf.NewTransport.
func helmGetterTransport(m dsl.Matcher) {
m.Match(`getter.WithTransport(&http.Transport{$*_})`).
Report(`getter.WithTransport called with a bare *http.Transport; use ssrf.NewTransport(tlsConfig) as Helm v4 bypasses http.DefaultTransport`)
}
// cloneDefaultTransport flags direct clones of *http.Transport outside main.go.
// The one legitimate clone is in main.go where http.DefaultTransport is globally
// wrapped with SSRF protection at server startup.
func cloneDefaultTransport(m dsl.Matcher) {
m.Match(`$_.(*http.Transport).Clone()`).
Where(!m.File().Name.Matches(`^main\.go$`)).
Report(`cloning *http.Transport directly is forbidden; use ssrf.NewTransport(tlsConfig) or ssrf.NewInternalTransport(tlsConfig) instead`)
}
// internalTransportMisuse flags calls to NewInternalTransport outside the proxy
// factory files where Chisel-tunnel and in-cluster K8s destinations are valid exemptions.
func internalTransportMisuse(m dsl.Matcher) {
m.Match(`ssrf.NewInternalTransport($*_)`).
Where(
!(m.File().PkgPath.Matches(`proxy/factory`) &&
m.File().Name.Matches(`^(docker|agent|local_transport|edge_transport|docker_unix|docker_windows)\.go$`))).
Report(`NewInternalTransport bypasses SSRF validation; only valid in the proxy factory files for local sockets and internally-routed endpoints`)
}
// dialerOverride flags direct assignments to any of the dialer fields on a transport.
// The only valid assignments are in docker_unix.go and docker_windows.go where a
// custom dialer is required for unix sockets and named pipes.
func dialerOverride(m dsl.Matcher) {
m.Match(`$_.DialContext = $*_`).
Where(
!(m.File().PkgPath.Matches(`proxy/factory`) &&
m.File().Name.Matches(`^(docker_unix|docker_windows)\.go$`))).
Report(`direct DialContext assignment replaces the transport dialer; use ssrf.NewTransport or ssrf.NewInternalTransport instead`)
m.Match(`$_.Dial = $*_`).
Where(
!(m.File().PkgPath.Matches(`proxy/factory`) &&
m.File().Name.Matches(`^(docker_unix|docker_windows)\.go$`))).
Report(`direct Dial assignment replaces the transport dialer; use ssrf.NewTransport or ssrf.NewInternalTransport instead`)
m.Match(`$_.DialTLSContext = $*_`).
Where(
!(m.File().PkgPath.Matches(`proxy/factory`) &&
m.File().Name.Matches(`^(docker_unix|docker_windows)\.go$`))).
Report(`direct DialTLSContext assignment replaces the transport dialer; use ssrf.NewTransport or ssrf.NewInternalTransport instead`)
m.Match(`$_.DialTLS = $*_`).
Where(
!(m.File().PkgPath.Matches(`proxy/factory`) &&
m.File().Name.Matches(`^(docker_unix|docker_windows)\.go$`))).
Report(`direct DialTLS assignment replaces the transport dialer; use ssrf.NewTransport or ssrf.NewInternalTransport instead`)
}

5
analysis/tools.go Normal file
View File

@@ -0,0 +1,5 @@
//go:build tools
package gorules
import _ "github.com/quasilyte/go-ruleguard/dsl"

1
api/.swaggo Normal file
View File

@@ -0,0 +1 @@
replace k8s.io/apimachinery/pkg/apis/meta/v1.Duration string

View File

@@ -1,6 +1,7 @@
package agent
import (
"context"
"crypto/tls"
"errors"
"fmt"
@@ -11,6 +12,7 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/url"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
"github.com/rs/zerolog/log"
)
@@ -19,10 +21,14 @@ import (
//
// it sends a ping to the agent and parses the version and platform from the headers
func GetAgentVersionAndPlatform(endpointUrl string, tlsConfig *tls.Config) (portainer.AgentPlatform, string, error) { //nolint:forbidigo
if err := ssrf.CheckURL(context.Background(), endpointUrl); err != nil {
return 0, "", err
}
httpCli := &http.Client{Timeout: 3 * time.Second}
if tlsConfig != nil {
httpCli.Transport = &http.Transport{TLSClientConfig: tlsConfig}
httpCli.Transport = ssrf.NewTransport(tlsConfig)
}
parsedURL, err := url.ParseURL(endpointUrl + "/ping")

View File

@@ -306,6 +306,10 @@ func (service *Service) snapshotAndLog(endpointID portainer.EndpointID, tunnelPo
Err(err).
Msg("unable to snapshot Edge environment")
if service.dataStore.IsErrObjectNotFound(err) {
service.close(endpointID)
}
return false
}

View File

@@ -187,6 +187,24 @@ func TestCheckTunnelsKeepsHasSnapshotFalseOnSnapshotFailure(t *testing.T) {
require.False(t, s.activeTunnels[endpoint.ID].HasSnapshot, "HasSnapshot must stay false after failure")
}
func TestCheckTunnelsClosesStaleEntryForDeletedEndpoint(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
// Endpoint is not created in the store, simulates deletion while tunnel stays open.
s := NewService(store, nil, nil)
s.activeTunnels[1] = &portainer.TunnelDetails{
Status: portainer.EdgeAgentManagementRequired,
Port: 50010,
LastActivity: time.Now(),
}
s.checkTunnels()
require.Nil(t, s.activeTunnels[1], "stale tunnel for deleted endpoint must be removed immediately")
}
func TestCheckTunnelsClosesIdleTunnelAndSnapshots(t *testing.T) {
t.Parallel()

View File

@@ -82,17 +82,24 @@ func (s *Service) Open(endpoint *portainer.Endpoint) error {
return nil
}
// close removes the tunnel from the map so the agent will close it
// close removes the tunnel from the map so the agent will close it.
// The lock is released before cleaning up the chisel user and proxy to avoid
// blocking Config/Open callers while DeleteUser interacts with chisel internals.
func (s *Service) close(endpointID portainer.EndpointID) {
s.mu.Lock()
defer s.mu.Unlock()
tun, ok := s.activeTunnels[endpointID]
if !ok {
s.mu.Unlock()
return
}
if len(tun.Credentials) > 0 && s.chiselServer != nil {
delete(s.activeTunnels, endpointID)
cache.Del(endpointID)
s.mu.Unlock()
if s.chiselServer != nil {
user, _, _ := strings.Cut(tun.Credentials, ":")
s.chiselServer.DeleteUser(user)
}
@@ -100,10 +107,6 @@ func (s *Service) close(endpointID portainer.EndpointID) {
if s.ProxyManager != nil {
s.ProxyManager.DeleteEndpointProxy(endpointID)
}
delete(s.activeTunnels, endpointID)
cache.Del(endpointID)
}
// Config returns the tunnel details needed for the agent to connect

View File

@@ -56,6 +56,8 @@ func CLIFlags() *portainer.CLIFlags {
TrustedOrigins: kingpin.Flag("trusted-origins", "List of trusted origins for CSRF protection. Separate multiple origins with a comma.").Envar(portainer.TrustedOriginsEnvVar).String(),
CSP: kingpin.Flag("csp", "Content Security Policy (CSP) header").Envar(portainer.CSPEnvVar).Default("true").Bool(),
CompactDB: kingpin.Flag("compact-db", "Enable database compaction on startup").Envar(portainer.CompactDBEnvVar).Default("false").Bool(),
NoSetupToken: kingpin.Flag("no-setup-token", "Disable the setup token requirement for admin initialization and restore on an uninitialized instance").Envar(portainer.NoSetupTokenEnvVar).Bool(),
SetupToken: kingpin.Flag("setup-token", "Set a custom setup token for admin initialization and restore on an uninitialized instance (overrides auto-generation)").Envar(portainer.SetupTokenEnvVar).String(),
}
}

View File

@@ -4,6 +4,7 @@ import (
"cmp"
"context"
"crypto/sha256"
nethttp "net/http"
"os"
"path"
"strings"
@@ -29,6 +30,7 @@ import (
"github.com/portainer/portainer/api/http"
"github.com/portainer/portainer/api/http/proxy"
kubeproxy "github.com/portainer/portainer/api/http/proxy/factory/kubernetes"
"github.com/portainer/portainer/api/http/security/setuptoken"
"github.com/portainer/portainer/api/internal/authorization"
"github.com/portainer/portainer/api/internal/edge/edgestacks"
"github.com/portainer/portainer/api/internal/endpointutils"
@@ -51,10 +53,15 @@ import (
"github.com/portainer/portainer/pkg/featureflags"
"github.com/portainer/portainer/pkg/fips"
"github.com/portainer/portainer/pkg/libhelm"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
"github.com/portainer/portainer/pkg/libstack/compose"
libswarm "github.com/portainer/portainer/pkg/libstack/swarm"
"github.com/portainer/portainer/pkg/validate"
gogitclient "github.com/go-git/go-git/v5/plumbing/transport/client"
gogitraw "github.com/go-git/go-git/v5/plumbing/transport/git"
gogithttp "github.com/go-git/go-git/v5/plumbing/transport/http"
gogitssh "github.com/go-git/go-git/v5/plumbing/transport/ssh"
"github.com/google/uuid"
"github.com/rs/zerolog/log"
)
@@ -225,6 +232,32 @@ func initSnapshotService(
return snapshotService, nil
}
func resolveSetupToken(tx dataservices.DataStoreTx, providedToken string) (string, error) {
admins, err := tx.User().UsersByRole(portainer.AdministratorRole)
if err != nil {
return "", err
}
if len(admins) > 0 {
return "", nil
}
if providedToken != "" {
log.Info().Msg("using custom setup token; admin initialization and backup restore require this token in the X-Setup-Token header")
return providedToken, nil
}
token, err := setuptoken.Generate()
if err != nil {
return "", err
}
log.Info().
Str("setup_token", token).
Msg("no administrator account configured; admin initialization and backup restore require this setup token in the X-Setup-Token header. Start with --no-setup-token to disable.")
return token, nil
}
func initStatus(instanceID string) *portainer.Status {
return &portainer.Status{
Version: portainer.APIVersion,
@@ -374,6 +407,19 @@ func buildServer(flags *portainer.CLIFlags, shutdownCtx context.Context, shutdow
log.Fatal().Msg("The database schema version does not align with the server version. Please consider reverting to the previous server version or addressing the database migration issue.")
}
if err := ssrf.Configure(dataStore.AllowList()); err != nil {
log.Fatal().Err(err).Msg("failed initializing ssrf service")
}
if !ssrf.WrapDefaultTransport() {
log.Fatal().Msg("failed to wrap default HTTP transport with SSRF protection")
}
gogithttp.DefaultClient = gogithttp.NewClient(&nethttp.Client{Transport: nethttp.DefaultTransport})
gogitclient.InstallProtocol("git", git.NewSSRFGitTransport(gogitraw.DefaultClient))
gogitclient.InstallProtocol("ssh", git.NewSSRFGitTransport(gogitssh.DefaultClient))
gogitclient.InstallProtocol("file", nil)
instanceID, err := dataStore.Version().InstanceID()
if err != nil {
log.Fatal().Err(err).Msg("failed getting instance id")
@@ -510,6 +556,17 @@ func buildServer(flags *portainer.CLIFlags, shutdownCtx context.Context, shutdow
}
}
setupToken := ""
if adminPasswordHash == "" && !*flags.NoSetupToken {
if err := dataStore.ViewTx(func(tx dataservices.DataStoreTx) error {
var txErr error
setupToken, txErr = resolveSetupToken(tx, *flags.SetupToken)
return txErr
}); err != nil {
log.Fatal().Err(err).Msg("failed initializing setup token")
}
}
if err := reverseTunnelService.StartTunnelServer(*flags.TunnelAddr, *flags.TunnelPort, snapshotService); err != nil {
log.Fatal().Err(err).Msg("failed starting tunnel server")
}
@@ -601,6 +658,7 @@ func buildServer(flags *portainer.CLIFlags, shutdownCtx context.Context, shutdow
PlatformService: platformService,
PullLimitCheckDisabled: *flags.PullLimitCheckDisabled,
TrustedOrigins: trustedOrigins,
SetupToken: setupToken,
}
}

View File

@@ -12,6 +12,44 @@ import (
"github.com/stretchr/testify/require"
)
func Test_resolveSetupToken(t *testing.T) {
t.Parallel()
t.Run("admin already exists — returns empty token", func(t *testing.T) {
admin := portainer.User{Role: portainer.AdministratorRole}
store := testhelpers.NewDatastore(testhelpers.WithUsers([]portainer.User{admin}))
token, err := resolveSetupToken(store, "")
require.NoError(t, err)
assert.Empty(t, token)
})
t.Run("no admin — generates a 64-char hex token", func(t *testing.T) {
store := testhelpers.NewDatastore(testhelpers.WithUsers([]portainer.User{}))
token, err := resolveSetupToken(store, "")
require.NoError(t, err)
assert.Len(t, token, 64)
token2, err := resolveSetupToken(store, "")
require.NoError(t, err)
assert.NotEqual(t, token, token2)
})
t.Run("no admin — uses provided token", func(t *testing.T) {
store := testhelpers.NewDatastore(testhelpers.WithUsers([]portainer.User{}))
token, err := resolveSetupToken(store, "mysecrettoken")
require.NoError(t, err)
assert.Equal(t, "mysecrettoken", token)
})
t.Run("admin already exists — ignores provided token", func(t *testing.T) {
admin := portainer.User{Role: portainer.AdministratorRole}
store := testhelpers.NewDatastore(testhelpers.WithUsers([]portainer.User{admin}))
token, err := resolveSetupToken(store, "mysecrettoken")
require.NoError(t, err)
assert.Empty(t, token)
})
}
const secretFileName = "secret.txt"
func createPasswordFile(t *testing.T, secretPath, password string) string {

View File

@@ -4,6 +4,7 @@ import (
"crypto/rand"
"errors"
"io"
"slices"
)
type Nonce struct {
@@ -45,7 +46,7 @@ func (n *Nonce) Value() []byte {
func (n *Nonce) Increment() error {
// Start incrementing from the least significant byte
for i := len(n.val) - 1; i >= 0; i-- {
for i := range slices.Backward(n.val) {
// Increment the current byte
n.val[i]++

View File

@@ -0,0 +1,131 @@
package allowlist
import (
"fmt"
lru "github.com/hashicorp/golang-lru"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
)
const (
BucketName = "allowlist"
)
type Service struct {
baseService dataservices.BaseDataService[portainer.AllowList, portainer.AllowListKey]
cache *lru.Cache
}
func (service *Service) BucketName() string {
return service.baseService.BucketName()
}
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
service := &Service{
baseService: dataservices.BaseDataService[portainer.AllowList, portainer.AllowListKey]{
Bucket: BucketName,
Connection: connection,
},
}
err = service.populateCache()
return service, err
}
func (service *Service) populateCache() error {
allowListKeys := []portainer.AllowListKey{portainer.AllowListSSRF}
cache, err := lru.New(len(allowListKeys))
if err != nil {
return err
}
for _, k := range allowListKeys {
allowList, err := service.baseService.Read(k)
if dataservices.IsErrObjectNotFound(err) {
allowList = &portainer.AllowList{
ID: k,
Mode: portainer.SSRFModeOff,
Entries: []string{},
}
} else if err != nil {
return err
}
parsedAllowList := ssrf.ParseAllowedHosts(allowList.Entries)
parsedAllowList.Mode = allowList.Mode
cache.Add(k, &parsedAllowList)
}
service.cache = cache
return nil
}
func (service *Service) Tx(tx portainer.Transaction) *ServiceTx {
return &ServiceTx{
baseService: service.baseService.Tx(tx),
cache: service.cache,
}
}
func (service *Service) Read(id portainer.AllowListKey) (*portainer.AllowList, error) {
var result *portainer.AllowList
if err := service.baseService.Connection.ViewTx(func(tx portainer.Transaction) error {
var err error
result, err = service.Tx(tx).Read(id)
return err
}); err != nil {
return nil, err
}
return result, nil
}
func (service *Service) ReadAll() ([]portainer.AllowList, error) {
var result []portainer.AllowList
if err := service.baseService.Connection.ViewTx(func(tx portainer.Transaction) error {
var err error
result, err = service.Tx(tx).ReadAll()
return err
}); err != nil {
return nil, err
}
return result, nil
}
func (service *Service) ReadParsed(id portainer.AllowListKey) (*portainer.ParsedAllowList, error) {
allowListAny, ok := service.cache.Get(id)
if ok {
allowList, ok := allowListAny.(*portainer.ParsedAllowList)
if !ok {
return nil, fmt.Errorf("expected ParsedAllowList in cache but got %T", allowListAny)
}
return allowList, nil
}
var result *portainer.ParsedAllowList
err := service.baseService.Connection.ViewTx(func(tx portainer.Transaction) error {
var err error
result, err = service.Tx(tx).ReadParsed(id)
return err
})
return result, err
}
func (service *Service) Update(id portainer.AllowListKey, allowList *portainer.AllowList) error {
return service.baseService.Connection.UpdateTx(func(tx portainer.Transaction) error {
return service.Tx(tx).Update(id, allowList)
})
}

View File

@@ -0,0 +1,89 @@
package allowlist_test
import (
"net"
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/datastore"
"github.com/stretchr/testify/require"
)
func TestAllowListReadEmpty(t *testing.T) {
t.Parallel()
_, ds := datastore.MustNewTestStore(t, false, false)
got, err := ds.AllowList().Read(portainer.AllowListSSRF)
expected := &portainer.AllowList{
ID: portainer.AllowListSSRF,
Mode: portainer.SSRFModeOff,
Entries: []string{},
}
require.NoError(t, err)
require.Equal(t, expected, got)
}
func TestAllowListUpdate(t *testing.T) {
t.Parallel()
_, ds := datastore.MustNewTestStore(t, false, false)
expected := &portainer.AllowList{
ID: portainer.AllowListSSRF,
Mode: portainer.SSRFModeEnforce,
Entries: []string{"example.com", "10.0.0.0/8"},
}
require.NoError(t, ds.AllowList().Update(portainer.AllowListSSRF, expected))
got, err := ds.AllowList().Read(portainer.AllowListSSRF)
require.NoError(t, err)
require.Equal(t, expected, got)
}
func TestAllowListReadAllEmpty(t *testing.T) {
t.Parallel()
_, ds := datastore.MustNewTestStore(t, false, false)
got, err := ds.AllowList().ReadAll()
require.NoError(t, err)
require.Equal(t, []portainer.AllowList{}, got)
}
func TestAllowListReadAllAfterUpdate(t *testing.T) {
t.Parallel()
_, ds := datastore.MustNewTestStore(t, false, false)
expected := portainer.AllowList{
ID: portainer.AllowListSSRF,
Mode: portainer.SSRFModeEnforce,
Entries: []string{"example.com", "10.0.0.0/8"},
}
require.NoError(t, ds.AllowList().Update(portainer.AllowListSSRF, &expected))
got, err := ds.AllowList().ReadAll()
require.NoError(t, err)
require.Equal(t, []portainer.AllowList{expected}, got)
}
func TestAllowListReadParsedAfterUpdate(t *testing.T) {
t.Parallel()
_, ds := datastore.MustNewTestStore(t, false, false)
require.NoError(t, ds.AllowList().Update(portainer.AllowListSSRF, &portainer.AllowList{
ID: portainer.AllowListSSRF,
Mode: portainer.SSRFModeEnforce,
Entries: []string{"example.com"},
}))
expected := &portainer.ParsedAllowList{
Mode: portainer.SSRFModeEnforce,
Nets: []*net.IPNet{},
Hosts: map[string]bool{
"example.com": true,
},
}
got, err := ds.AllowList().ReadParsed(portainer.AllowListSSRF)
require.NoError(t, err)
require.Equal(t, expected, got)
}

View File

@@ -0,0 +1,77 @@
package allowlist
import (
"fmt"
lru "github.com/hashicorp/golang-lru"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
)
type ServiceTx struct {
baseService dataservices.BaseDataServiceTx[portainer.AllowList, portainer.AllowListKey]
cache *lru.Cache
}
func (service *ServiceTx) BucketName() string {
return service.baseService.BucketName()
}
func (service *ServiceTx) ReadParsed(id portainer.AllowListKey) (*portainer.ParsedAllowList, error) {
allowListAny, ok := service.cache.Get(id)
if ok {
allowList, ok := allowListAny.(*portainer.ParsedAllowList)
if !ok {
return nil, fmt.Errorf("expected ParsedAllowList in cache but got %T", allowListAny)
}
return allowList, nil
}
allowList, err := service.Read(id)
if err != nil {
return nil, err
}
parsed := ssrf.ParseAllowedHosts(allowList.Entries)
parsed.Mode = allowList.Mode
service.cache.Add(id, &parsed)
return &parsed, nil
}
func (service *ServiceTx) Read(id portainer.AllowListKey) (*portainer.AllowList, error) {
allowList, err := service.baseService.Read(id)
if dataservices.IsErrObjectNotFound(err) {
allowList = &portainer.AllowList{
ID: id,
Mode: portainer.SSRFModeOff,
Entries: []string{},
}
} else if err != nil {
return nil, err
}
return allowList, nil
}
func (service *ServiceTx) ReadAll() ([]portainer.AllowList, error) {
allowLists, err := service.baseService.ReadAll()
if err != nil && !dataservices.IsErrObjectNotFound(err) {
return nil, err
}
return allowLists, nil
}
func (service *ServiceTx) Update(id portainer.AllowListKey, allowList *portainer.AllowList) error {
if err := service.baseService.Update(id, allowList); err != nil {
return err
}
parsed := ssrf.ParseAllowedHosts(allowList.Entries)
parsed.Mode = allowList.Mode
service.cache.Add(id, &parsed)
return nil
}

View File

@@ -0,0 +1,92 @@
package allowlist_test
import (
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/datastore"
"github.com/stretchr/testify/require"
)
func TestAllowListReadTx(t *testing.T) {
t.Parallel()
_, ds := datastore.MustNewTestStore(t, false, false)
var got *portainer.AllowList
require.NoError(t, ds.ViewTx(func(tx dataservices.DataStoreTx) error {
var err error
got, err = tx.AllowList().Read(portainer.AllowListSSRF)
return err
}))
expected := &portainer.AllowList{
ID: portainer.AllowListSSRF,
Mode: portainer.SSRFModeOff,
Entries: []string{},
}
require.Equal(t, expected, got)
}
func TestAllowListReadAllEmptyTx(t *testing.T) {
t.Parallel()
_, ds := datastore.MustNewTestStore(t, false, false)
var got []portainer.AllowList
require.NoError(t, ds.ViewTx(func(tx dataservices.DataStoreTx) error {
var err error
got, err = tx.AllowList().ReadAll()
return err
}))
require.Equal(t, []portainer.AllowList{}, got)
}
func TestAllowListReadAllAfterUpdateTx(t *testing.T) {
t.Parallel()
_, ds := datastore.MustNewTestStore(t, false, false)
expected := portainer.AllowList{
ID: portainer.AllowListSSRF,
Mode: portainer.SSRFModeEnforce,
Entries: []string{"example.com"},
}
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.AllowList().Update(portainer.AllowListSSRF, &expected)
}))
var got []portainer.AllowList
require.NoError(t, ds.ViewTx(func(tx dataservices.DataStoreTx) error {
var err error
got, err = tx.AllowList().ReadAll()
return err
}))
require.Equal(t, []portainer.AllowList{expected}, got)
}
func TestAllowListUpdateTx(t *testing.T) {
t.Parallel()
_, ds := datastore.MustNewTestStore(t, false, false)
expected := &portainer.AllowList{
ID: portainer.AllowListSSRF,
Mode: portainer.SSRFModeEnforce,
Entries: []string{"example.com"},
}
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.AllowList().Update(portainer.AllowListSSRF, expected)
}))
var got *portainer.AllowList
require.NoError(t, ds.ViewTx(func(tx dataservices.DataStoreTx) error {
var err error
got, err = tx.AllowList().Read(portainer.AllowListSSRF)
return err
}))
require.Equal(t, expected, got)
}

View File

@@ -8,6 +8,7 @@ import (
type (
DataStoreTx interface {
IsErrObjectNotFound(err error) bool
AllowList() AllowListService
CustomTemplate() CustomTemplateService
EdgeGroup() EdgeGroupService
EdgeJob() EdgeJobService
@@ -53,6 +54,15 @@ type (
DataStoreTx
}
// AllowListService represents a service for managing the URL allow list
AllowListService interface {
Read(id portainer.AllowListKey) (*portainer.AllowList, error)
ReadAll() ([]portainer.AllowList, error)
ReadParsed(id portainer.AllowListKey) (*portainer.ParsedAllowList, error)
Update(id portainer.AllowListKey, allowList *portainer.AllowList) error
BucketName() string
}
// CustomTemplateService represents a service to manage custom templates
CustomTemplateService interface {
BaseCRUD[portainer.CustomTemplate, portainer.CustomTemplateID]

View File

@@ -88,6 +88,7 @@ func (store *Store) newMigratorParameters(version *models.Version, flags *portai
EdgeGroupService: store.EdgeGroupService,
TunnelServerService: store.TunnelServerService,
PendingActionsService: store.PendingActionsService,
CustomTemplateService: store.CustomTemplateService,
SourceService: store.SourceService,
WorkflowService: store.WorkflowService,
}

View File

@@ -24,7 +24,6 @@ type legacyGitAuthentication struct {
Password string
Provider int `json:",omitempty"`
AuthorizationType int `json:",omitempty"`
GitCredentialID int
}
func (lrc *legacyRepoConfig) toRepoConfig() *gittypes.RepoConfig {
@@ -41,12 +40,6 @@ func (lrc *legacyRepoConfig) toRepoConfig() *gittypes.RepoConfig {
}
if lrc.Authentication != nil {
if lrc.Authentication.GitCredentialID != 0 {
log.Warn().
Int("git_credential_id", lrc.Authentication.GitCredentialID).
Msg("stack has a GitCredentialID reference which is not supported in CE; credential reference will be dropped during migration")
}
cfg.Authentication = &gittypes.GitAuthentication{
Username: lrc.Authentication.Username,
Password: lrc.Authentication.Password,
@@ -112,8 +105,8 @@ func (m *Migrator) migrateGitConfigToSources_2_43_0() error {
sourcesByKey := make(map[sourceDedupeKey]portainer.SourceID, len(existingSources))
for _, src := range existingSources {
if src.GitConfig != nil {
sourcesByKey[gitSourceKey(src.GitConfig)] = src.ID
if src.Git != nil {
sourcesByKey[gitSourceKey(src.Git)] = src.ID
}
}
@@ -133,9 +126,9 @@ func (m *Migrator) migrateGitConfigToSources_2_43_0() error {
if !exists {
src := &portainer.Source{
Name: gittypes.RepoName(cfg.URL),
Type: portainer.SourceTypeGit,
GitConfig: cfg,
Name: gittypes.RepoName(cfg.URL),
Type: portainer.SourceTypeGit,
Git: cfg,
}
if err := m.sourceService.Tx(tx).Create(src); err != nil {
return fmt.Errorf("failed to create source for stack %d: %w", ls.ID, err)
@@ -151,14 +144,14 @@ func (m *Migrator) migrateGitConfigToSources_2_43_0() error {
wf := &portainer.Workflow{
Name: liveStack.Name,
Artifacts: []portainer.ArtifactSources{{
Artifact: portainer.Artifact{
ReferenceName: cfg.ReferenceName,
ConfigFilePath: cfg.ConfigFilePath,
ConfigHash: cfg.ConfigHash,
StackID: portainer.StackID(ls.ID),
},
SourceIDs: []portainer.SourceID{srcID},
Artifacts: []portainer.Artifact{{
StackID: portainer.StackID(ls.ID),
Files: []portainer.ArtifactFile{{
SourceID: srcID,
Path: cfg.ConfigFilePath,
Ref: cfg.ReferenceName,
Hash: cfg.ConfigHash,
}},
}},
}
if err := m.workflowService.Tx(tx).Create(wf); err != nil {
@@ -180,3 +173,78 @@ func (m *Migrator) migrateGitConfigToSources_2_43_0() error {
return nil
}
func (m *Migrator) migrateCustomTemplateGitConfigToSources_2_43_0() error {
log.Info().Msg("migrating git-backed custom templates to Source records")
templates, err := m.customTemplateService.ReadAll()
if err != nil {
return err
}
existingSources, err := m.sourceService.ReadAll()
if err != nil {
return err
}
sourcesByKey := make(map[sourceDedupeKey]portainer.SourceID, len(existingSources))
for _, src := range existingSources {
if src.Git != nil {
sourcesByKey[gitSourceKey(src.Git)] = src.ID
}
}
for i := range templates {
t := &templates[i]
if t.GitConfig == nil || t.Artifact != nil {
continue
}
cfg := &gittypes.RepoConfig{
URL: gittypes.SanitizeURL(t.GitConfig.URL),
Authentication: t.GitConfig.Authentication,
TLSSkipVerify: t.GitConfig.TLSSkipVerify,
}
key := gitSourceKey(cfg)
var newSrcID portainer.SourceID
if err := m.stackService.Connection.UpdateTx(func(tx portainer.Transaction) error {
srcID, exists := sourcesByKey[key]
if !exists {
src := &portainer.Source{
Name: gittypes.RepoName(cfg.URL),
Type: portainer.SourceTypeGit,
Git: cfg,
}
if err := m.sourceService.Tx(tx).Create(src); err != nil {
return fmt.Errorf("failed to create source for custom template %d: %w", t.ID, err)
}
srcID = src.ID
newSrcID = src.ID
}
t.Artifact = &portainer.Artifact{
Files: []portainer.ArtifactFile{{
SourceID: srcID,
Path: t.GitConfig.ConfigFilePath,
Ref: t.GitConfig.ReferenceName,
Hash: t.GitConfig.ConfigHash,
}},
}
t.GitConfig = nil
return m.customTemplateService.Tx(tx).Update(t.ID, t)
}); err != nil {
return fmt.Errorf("failed to migrate custom template %d: %w", t.ID, err)
}
if newSrcID != 0 {
sourcesByKey[key] = newSrcID
}
}
return nil
}

View File

@@ -5,6 +5,7 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/database/boltdb"
"github.com/portainer/portainer/api/dataservices/customtemplate"
"github.com/portainer/portainer/api/dataservices/source"
"github.com/portainer/portainer/api/dataservices/stack"
"github.com/portainer/portainer/api/dataservices/workflow"
@@ -58,13 +59,13 @@ func TestMigrateGitConfigToSources_2_43_0_GitStackMigrated(t *testing.T) {
wf, err := workflowSvc.Read(migrated.WorkflowID)
require.NoError(t, err)
require.Len(t, wf.Artifacts, 1)
require.Len(t, wf.Artifacts[0].SourceIDs, 1)
require.Len(t, wf.Artifacts[0].Files, 1)
src, err := sourceSvc.Read(wf.Artifacts[0].SourceIDs[0])
src, err := sourceSvc.Read(wf.Artifacts[0].Files[0].SourceID)
require.NoError(t, err)
require.Equal(t, portainer.SourceTypeGit, src.Type)
require.Equal(t, gitStack.GitConfig.URL, src.GitConfig.URL)
require.Equal(t, gitStack.GitConfig.ReferenceName, src.GitConfig.ReferenceName)
require.Equal(t, gitStack.GitConfig.URL, src.Git.URL)
require.Equal(t, gitStack.GitConfig.ReferenceName, src.Git.ReferenceName)
}
func TestMigrateGitConfigToSources_2_43_0_NonGitStackUntouched(t *testing.T) {
@@ -170,8 +171,8 @@ func TestMigrateGitConfigToSources_2_43_0_DuplicateSourcesDeduped(t *testing.T)
sharedSourceID := sources[0].ID
for _, wf := range workflows {
require.Len(t, wf.Artifacts, 1)
require.Len(t, wf.Artifacts[0].SourceIDs, 1)
require.Equal(t, sharedSourceID, wf.Artifacts[0].SourceIDs[0])
require.Len(t, wf.Artifacts[0].Files, 1)
require.Equal(t, sharedSourceID, wf.Artifacts[0].Files[0].SourceID)
}
}
@@ -221,3 +222,241 @@ func TestMigrateGitConfigToSources_2_43_0_Idempotent(t *testing.T) {
require.NoError(t, err)
require.Len(t, workflows, 1)
}
func TestMigrateCustomTemplateGitConfigToSources_2_43_0_GitTemplateMigrated(t *testing.T) {
t.Parallel()
conn := &boltdb.DbConnection{Path: t.TempDir()}
err := conn.Open()
require.NoError(t, err)
defer logs.CloseAndLogErr(conn)
stackSvc, err := stack.NewService(conn)
require.NoError(t, err)
sourceSvc, err := source.NewService(conn)
require.NoError(t, err)
customTemplateSvc, err := customtemplate.NewService(conn)
require.NoError(t, err)
m := NewMigrator(&MigratorParameters{
StackService: stackSvc,
SourceService: sourceSvc,
CustomTemplateService: customTemplateSvc,
})
tmpl := &portainer.CustomTemplate{
ID: 1,
GitConfig: &gittypes.RepoConfig{
URL: "https://github.com/example/repo",
ReferenceName: "refs/heads/main",
ConfigFilePath: "docker-compose.yml",
ConfigHash: "abc123",
},
}
err = conn.CreateObjectWithId(customtemplate.BucketName, int(tmpl.ID), tmpl)
require.NoError(t, err)
err = m.migrateCustomTemplateGitConfigToSources_2_43_0()
require.NoError(t, err)
migrated, err := customTemplateSvc.Read(tmpl.ID)
require.NoError(t, err)
require.NotNil(t, migrated.Artifact)
require.Nil(t, migrated.GitConfig)
require.Len(t, migrated.Artifact.Files, 1)
require.Equal(t, "refs/heads/main", migrated.Artifact.Files[0].Ref)
require.Equal(t, "docker-compose.yml", migrated.Artifact.Files[0].Path)
require.Equal(t, "abc123", migrated.Artifact.Files[0].Hash)
src, err := sourceSvc.Read(migrated.Artifact.Files[0].SourceID)
require.NoError(t, err)
require.Equal(t, portainer.SourceTypeGit, src.Type)
require.Equal(t, "https://github.com/example/repo", src.Git.URL)
}
func TestMigrateCustomTemplateGitConfigToSources_2_43_0_NonGitTemplateUntouched(t *testing.T) {
t.Parallel()
conn := &boltdb.DbConnection{Path: t.TempDir()}
err := conn.Open()
require.NoError(t, err)
defer logs.CloseAndLogErr(conn)
stackSvc, err := stack.NewService(conn)
require.NoError(t, err)
sourceSvc, err := source.NewService(conn)
require.NoError(t, err)
customTemplateSvc, err := customtemplate.NewService(conn)
require.NoError(t, err)
m := NewMigrator(&MigratorParameters{
StackService: stackSvc,
SourceService: sourceSvc,
CustomTemplateService: customTemplateSvc,
})
tmpl := &portainer.CustomTemplate{ID: 1, Title: "plain-template"}
err = conn.CreateObjectWithId(customtemplate.BucketName, int(tmpl.ID), tmpl)
require.NoError(t, err)
err = m.migrateCustomTemplateGitConfigToSources_2_43_0()
require.NoError(t, err)
result, err := customTemplateSvc.Read(tmpl.ID)
require.NoError(t, err)
require.Nil(t, result.Artifact)
require.Nil(t, result.GitConfig)
sources, err := sourceSvc.ReadAll()
require.NoError(t, err)
require.Empty(t, sources)
}
func TestMigrateCustomTemplateGitConfigToSources_2_43_0_AlreadyMigratedSkipped(t *testing.T) {
t.Parallel()
conn := &boltdb.DbConnection{Path: t.TempDir()}
err := conn.Open()
require.NoError(t, err)
defer logs.CloseAndLogErr(conn)
stackSvc, err := stack.NewService(conn)
require.NoError(t, err)
sourceSvc, err := source.NewService(conn)
require.NoError(t, err)
customTemplateSvc, err := customtemplate.NewService(conn)
require.NoError(t, err)
m := NewMigrator(&MigratorParameters{
StackService: stackSvc,
SourceService: sourceSvc,
CustomTemplateService: customTemplateSvc,
})
// Template already has Artifact set (already migrated)
srcID := portainer.SourceID(99)
tmpl := &portainer.CustomTemplate{
ID: 1,
GitConfig: &gittypes.RepoConfig{
URL: "https://github.com/example/repo",
},
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{{SourceID: srcID}},
},
}
err = conn.CreateObjectWithId(customtemplate.BucketName, int(tmpl.ID), tmpl)
require.NoError(t, err)
err = m.migrateCustomTemplateGitConfigToSources_2_43_0()
require.NoError(t, err)
sources, err := sourceSvc.ReadAll()
require.NoError(t, err)
require.Empty(t, sources, "no new sources should be created for already-migrated templates")
}
func TestMigrateCustomTemplateGitConfigToSources_2_43_0_DuplicateSourcesDeduped(t *testing.T) {
t.Parallel()
conn := &boltdb.DbConnection{Path: t.TempDir()}
err := conn.Open()
require.NoError(t, err)
defer logs.CloseAndLogErr(conn)
stackSvc, err := stack.NewService(conn)
require.NoError(t, err)
sourceSvc, err := source.NewService(conn)
require.NoError(t, err)
customTemplateSvc, err := customtemplate.NewService(conn)
require.NoError(t, err)
m := NewMigrator(&MigratorParameters{
StackService: stackSvc,
SourceService: sourceSvc,
CustomTemplateService: customTemplateSvc,
})
sharedURL := "https://github.com/example/shared-repo"
tmpl1 := &portainer.CustomTemplate{
ID: 1,
Title: "template-a",
GitConfig: &gittypes.RepoConfig{
URL: sharedURL,
ReferenceName: "refs/heads/main",
},
}
tmpl2 := &portainer.CustomTemplate{
ID: 2,
Title: "template-b",
GitConfig: &gittypes.RepoConfig{
URL: sharedURL,
ReferenceName: "refs/heads/develop",
},
}
err = conn.CreateObjectWithId(customtemplate.BucketName, int(tmpl1.ID), tmpl1)
require.NoError(t, err)
err = conn.CreateObjectWithId(customtemplate.BucketName, int(tmpl2.ID), tmpl2)
require.NoError(t, err)
err = m.migrateCustomTemplateGitConfigToSources_2_43_0()
require.NoError(t, err)
sources, err := sourceSvc.ReadAll()
require.NoError(t, err)
require.Len(t, sources, 1, "two templates with the same URL must share one Source")
sharedSrcID := sources[0].ID
migrated1, err := customTemplateSvc.Read(tmpl1.ID)
require.NoError(t, err)
require.NotNil(t, migrated1.Artifact)
require.Equal(t, sharedSrcID, migrated1.Artifact.Files[0].SourceID)
migrated2, err := customTemplateSvc.Read(tmpl2.ID)
require.NoError(t, err)
require.NotNil(t, migrated2.Artifact)
require.Equal(t, sharedSrcID, migrated2.Artifact.Files[0].SourceID)
}
func TestMigrateCustomTemplateGitConfigToSources_2_43_0_Idempotent(t *testing.T) {
t.Parallel()
conn := &boltdb.DbConnection{Path: t.TempDir()}
err := conn.Open()
require.NoError(t, err)
defer logs.CloseAndLogErr(conn)
stackSvc, err := stack.NewService(conn)
require.NoError(t, err)
sourceSvc, err := source.NewService(conn)
require.NoError(t, err)
customTemplateSvc, err := customtemplate.NewService(conn)
require.NoError(t, err)
m := NewMigrator(&MigratorParameters{
StackService: stackSvc,
SourceService: sourceSvc,
CustomTemplateService: customTemplateSvc,
})
tmpl := &portainer.CustomTemplate{
ID: 1,
GitConfig: &gittypes.RepoConfig{
URL: "https://github.com/example/repo",
},
}
err = conn.CreateObjectWithId(customtemplate.BucketName, int(tmpl.ID), tmpl)
require.NoError(t, err)
err = m.migrateCustomTemplateGitConfigToSources_2_43_0()
require.NoError(t, err)
// Second run must not create duplicate Source records
err = m.migrateCustomTemplateGitConfigToSources_2_43_0()
require.NoError(t, err)
sources, err := sourceSvc.ReadAll()
require.NoError(t, err)
require.Len(t, sources, 1)
}

View File

@@ -5,6 +5,7 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/database/models"
"github.com/portainer/portainer/api/dataservices/customtemplate"
"github.com/portainer/portainer/api/dataservices/dockerhub"
"github.com/portainer/portainer/api/dataservices/edgegroup"
"github.com/portainer/portainer/api/dataservices/edgejob"
@@ -66,6 +67,7 @@ type (
edgeGroupService *edgegroup.Service
TunnelServerService *tunnelserver.Service
pendingActionsService *pendingactions.Service
customTemplateService *customtemplate.Service
sourceService *source.Service
workflowService *workflow.Service
}
@@ -98,6 +100,7 @@ type (
EdgeGroupService *edgegroup.Service
TunnelServerService *tunnelserver.Service
PendingActionsService *pendingactions.Service
CustomTemplateService *customtemplate.Service
SourceService *source.Service
WorkflowService *workflow.Service
}
@@ -132,6 +135,7 @@ func NewMigrator(parameters *MigratorParameters) *Migrator {
edgeGroupService: parameters.EdgeGroupService,
TunnelServerService: parameters.TunnelServerService,
pendingActionsService: parameters.PendingActionsService,
customTemplateService: parameters.CustomTemplateService,
sourceService: parameters.SourceService,
workflowService: parameters.WorkflowService,
}
@@ -268,7 +272,10 @@ func (m *Migrator) initMigrations() {
m.addMigrations("2.40.0", m.migrateRegistryAccessSASecrets_2_40_0)
m.addMigrations("2.43.0", m.migrateGitConfigToSources_2_43_0)
m.addMigrations("2.43.0",
m.migrateGitConfigToSources_2_43_0,
m.migrateCustomTemplateGitConfigToSources_2_43_0,
)
// WARNING: do not change migrations that have already been released!

View File

@@ -7,6 +7,7 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/database/models"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/dataservices/allowlist"
"github.com/portainer/portainer/api/dataservices/apikeyrepository"
"github.com/portainer/portainer/api/dataservices/customtemplate"
"github.com/portainer/portainer/api/dataservices/dockerhub"
@@ -51,6 +52,7 @@ type Store struct {
connection portainer.Connection
fileService portainer.FileService
AllowListService *allowlist.Service
CustomTemplateService *customtemplate.Service
DockerHubService *dockerhub.Service
EdgeGroupService *edgegroup.Service
@@ -84,6 +86,12 @@ type Store struct {
}
func (store *Store) initServices() error {
allowListService, err := allowlist.NewService(store.connection)
if err != nil {
return err
}
store.AllowListService = allowListService
authorizationsetService, err := role.NewService(store.connection)
if err != nil {
return err
@@ -275,6 +283,11 @@ func (store *Store) PendingActions() dataservices.PendingActionsService {
return store.PendingActionsService
}
// AllowList gives access to the AllowList data management layer
func (store *Store) AllowList() dataservices.AllowListService {
return store.AllowListService
}
// CustomTemplate gives access to the CustomTemplate data management layer
func (store *Store) CustomTemplate() dataservices.CustomTemplateService {
return store.CustomTemplateService
@@ -654,7 +667,7 @@ func (store *Store) Export(filename string) (err error) {
return err
}
return os.WriteFile(filename, b, 0600)
return os.WriteFile(filename, b, 0o600)
}
func (store *Store) Import(filename string) (err error) {

View File

@@ -14,6 +14,10 @@ func (tx *StoreTx) IsErrObjectNotFound(err error) bool {
return tx.store.IsErrObjectNotFound(err)
}
func (tx *StoreTx) AllowList() dataservices.AllowListService {
return tx.store.AllowListService.Tx(tx.tx)
}
func (tx *StoreTx) CustomTemplate() dataservices.CustomTemplateService {
return tx.store.CustomTemplateService.Tx(tx.tx)
}

View File

@@ -1,4 +1,5 @@
{
"allowlist": null,
"api_key": null,
"customtemplates": null,
"dockerhub": [
@@ -920,7 +921,7 @@
}
],
"version": {
"VERSION": "{\"SchemaVersion\":\"2.43.0\",\"MigratorCount\":1,\"Edition\":1,\"InstanceID\":\"463d5c47-0ea5-4aca-85b1-405ceefee254\"}"
"VERSION": "{\"SchemaVersion\":\"2.43.0\",\"MigratorCount\":2,\"Edition\":1,\"InstanceID\":\"463d5c47-0ea5-4aca-85b1-405ceefee254\"}"
},
"webhooks": null,
"workflows": null

View File

@@ -11,6 +11,8 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/crypto"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
"github.com/rs/zerolog/log"
"github.com/docker/docker/api/types/image"
@@ -88,7 +90,7 @@ func createTCPClient(endpoint *portainer.Endpoint, timeout *time.Duration) (*cli
client.WithHTTPClient(httpCli),
}
if nnTransport, ok := httpCli.Transport.(*NodeNameTransport); ok && nnTransport.TLSClientConfig != nil {
if endpoint.TLSConfig.TLS {
opts = append(opts, client.WithScheme("https"))
}
@@ -122,7 +124,7 @@ func createAgentClient(endpoint *portainer.Endpoint, endpointURL string, signatu
client.WithHTTPHeaders(headers),
}
if nnTransport, ok := httpCli.Transport.(*NodeNameTransport); ok && nnTransport.TLSClientConfig != nil {
if endpoint.TLSConfig.TLS {
opts = append(opts, client.WithScheme("https"))
}
@@ -184,17 +186,20 @@ func (t *NodeNameTransport) RoundTrip(req *http.Request) (*http.Response, error)
}
func httpClient(endpoint *portainer.Endpoint, timeout *time.Duration) (*http.Client, error) {
transport := &NodeNameTransport{
Transport: &http.Transport{},
}
var transport *NodeNameTransport
if endpoint.TLSConfig.TLS {
tlsConfig, err := crypto.CreateTLSConfigurationFromDisk(endpoint.TLSConfig)
if err != nil {
return nil, err
}
transport.TLSClientConfig = tlsConfig
transport = &NodeNameTransport{
Transport: ssrf.NewTransport(tlsConfig),
}
} else {
transport = &NodeNameTransport{
Transport: ssrf.NewTransport(nil),
}
}
clientTimeout := defaultDockerRequestTimeout

View File

@@ -26,7 +26,12 @@ func NewRegistryClient(dataStore dataservices.DataStore) *RegistryClient {
func (c *RegistryClient) RegistryAuth(image Image) (string, string, error) {
registry, err := cachedRegistry(image.Opts.Name)
if err != nil {
registries, err := c.dataStore.Registry().ReadAll()
var registries []portainer.Registry
err = c.dataStore.ViewTx(func(tx dataservices.DataStoreTx) error {
var err error
registries, err = tx.Registry().ReadAll()
return err
})
if err != nil {
return "", "", err
}
@@ -59,7 +64,12 @@ func (c *RegistryClient) CertainRegistryAuth(registry *portainer.Registry) (stri
func (c *RegistryClient) EncodedRegistryAuth(image Image) (string, error) {
registry, err := cachedRegistry(image.Opts.Name)
if err != nil {
registries, err := c.dataStore.Registry().ReadAll()
var registries []portainer.Registry
err = c.dataStore.ViewTx(func(tx dataservices.DataStoreTx) error {
var err error
registries, err = tx.Registry().ReadAll()
return err
})
if err != nil {
return "", err
}

View File

@@ -38,7 +38,7 @@ func fetchEndpointProxy(proxyManager *proxy.Manager, endpoint *portainer.Endpoin
// portainerRegistriesToAuthConfigs converts registries to Docker auth configs.
// Callers must ensure ECR tokens are valid before calling this function (e.g. via
// registryutils.ValidateRegistriesECRTokens with a real DataStoreTx). This function
// registryutils.RefreshAndPersistECRTokens with a real DataStoreTx). This function
// intentionally performs no DB writes to avoid write-lock contention when called inside
// an active BoltDB write transaction.
func portainerRegistriesToAuthConfigs(registries []portainer.Registry) []types.AuthConfig {

View File

@@ -89,7 +89,7 @@ func JoinPaths(trustedRoot string, untrustedPaths ...string) string {
trustedRoot = "."
}
p := filepath.Join(trustedRoot, filepath.Join(append([]string{"/"}, untrustedPaths...)...)) //nolint:forbidigo
p := filepath.Join(trustedRoot, filepath.Join(append([]string{"/"}, untrustedPaths...)...))
// avoid setting a volume name from the untrusted paths
vnp := filepath.VolumeName(p)

View File

@@ -14,12 +14,13 @@ import (
"github.com/portainer/portainer/api/crypto"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/logs"
"github.com/rs/zerolog/log"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
"github.com/go-git/go-git/v5"
"github.com/go-git/go-git/v5/plumbing/filemode"
githttp "github.com/go-git/go-git/v5/plumbing/transport/http"
"github.com/pkg/errors"
"github.com/rs/zerolog/log"
"github.com/segmentio/encoding/json"
)
@@ -64,15 +65,10 @@ func NewAzureClient() *azureClient {
}
func newHttpClientForAzure(insecureSkipVerify bool) *http.Client {
httpsCli := &http.Client{
Transport: &http.Transport{
TLSClientConfig: crypto.CreateTLSConfiguration(insecureSkipVerify),
Proxy: http.ProxyFromEnvironment,
},
Timeout: 300 * time.Second,
return &http.Client{
Transport: ssrf.NewTransport(crypto.CreateTLSConfiguration(insecureSkipVerify)),
Timeout: 300 * time.Second,
}
return httpsCli
}
func (a *azureClient) Download(ctx context.Context, destination string, opt *git.CloneOptions) error {

View File

@@ -2,18 +2,24 @@ package git
import (
"context"
"os"
"path/filepath"
"strings"
"github.com/portainer/portainer/api/filesystem"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/go-git/go-billy/v5"
"github.com/go-git/go-billy/v5/osfs"
"github.com/go-git/go-git/v5"
"github.com/go-git/go-git/v5/config"
"github.com/go-git/go-git/v5/plumbing/cache"
"github.com/go-git/go-git/v5/plumbing/filemode"
"github.com/go-git/go-git/v5/plumbing/object"
gogitfs "github.com/go-git/go-git/v5/storage/filesystem"
"github.com/go-git/go-git/v5/storage/memory"
"github.com/pkg/errors"
"github.com/rs/zerolog/log"
)
// noSymlinkFS wraps a billy.Filesystem and rejects symlink creation to prevent
@@ -42,28 +48,35 @@ func NewGitClient(preserveGitDir bool) *gitClient {
}
func (c *gitClient) Download(ctx context.Context, dst string, opt *git.CloneOptions) error {
if c.preserveGitDirectory {
_, err := git.PlainCloneContext(ctx, dst, false, opt)
if err != nil {
if err.Error() == "authentication required" {
return gittypes.ErrAuthenticationFailure
}
return errors.Wrap(err, "failed to clone git repository")
}
return nil
resolved, err := filepath.EvalSymlinks(dst)
if err != nil && !errors.Is(err, os.ErrNotExist) {
return errors.Wrap(err, "failed to resolve destination path")
}
if err == nil {
dst = resolved
}
// Memory storage avoids a macOS filesystem conflict where go-git's init
// creates dst/.git as a directory before checkout, causing EISDIR errors
// that mask ErrSymlinkDetected from noSymlinkFS.
wt := NewNoSymlinkFS(osfs.New(dst))
_, err := git.CloneContext(ctx, memory.NewStorage(), wt, opt)
dot := osfs.New(filesystem.JoinPaths(dst, ".git"))
storer := gogitfs.NewStorage(dot, cache.NewObjectLRU(0))
_, err = git.CloneContext(ctx, storer, wt, opt)
if err != nil {
if err.Error() == "authentication required" {
return gittypes.ErrAuthenticationFailure
}
return errors.Wrap(err, "failed to clone git repository")
}
if c.preserveGitDirectory {
return nil
}
if err := os.RemoveAll(filesystem.JoinPaths(dst, ".git")); err != nil {
log.Error().Err(err).Msg("failed to remove .git directory")
}
return nil
}
@@ -105,7 +118,7 @@ func (c *gitClient) ListRefs(ctx context.Context, repositoryUrl string, opt *git
URLs: []string{repositoryUrl},
})
refs, err := rem.List(opt)
refs, err := rem.ListContext(ctx, opt)
if err != nil {
return nil, checkGitError(err)
}

View File

@@ -99,6 +99,19 @@ func Test_ClonePublicRepository_NoGitDirectory(t *testing.T) {
assert.NoDirExists(t, filesystem.JoinPaths(dir, ".git"))
}
func Test_ClonePublicRepository_NonExistentDst(t *testing.T) {
t.Parallel()
service := Service{git: NewGitClient(false)}
repositoryURL := setup(t)
referenceName := "refs/heads/main"
dir := filesystem.JoinPaths(t.TempDir(), "sub", "dir")
err := service.CloneRepository(t.Context(), dir, repositoryURL, referenceName, "", "", false)
require.NoError(t, err)
assert.DirExists(t, dir)
assert.NoDirExists(t, filesystem.JoinPaths(dir, ".git"))
}
func Test_latestCommitID(t *testing.T) {
t.Parallel()
service := Service{git: NewGitClient(true)} // no need for http client since the test access the repo via file system.
@@ -262,6 +275,7 @@ func createBareRepoWithSymlink(t *testing.T) string {
}
func Test_Download_RejectsSymlink(t *testing.T) {
t.Parallel()
client := NewGitClient(false)
repoURL := createBareRepoWithSymlink(t)

53
api/git/ssrf_transport.go Normal file
View File

@@ -0,0 +1,53 @@
package git
import (
"context"
"fmt"
"net"
"strconv"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
gittransport "github.com/go-git/go-git/v5/plumbing/transport"
)
const gitDefaultPort = 9418
// ssrfGitTransport wraps a git:// transport and validates the resolved IP
// against the SSRF policy before establishing connections.
type ssrfGitTransport struct {
inner gittransport.Transport
}
// NewSSRFGitTransport wraps inner and blocks connections to private IP ranges
// according to the active SSRF policy.
func NewSSRFGitTransport(inner gittransport.Transport) gittransport.Transport {
return &ssrfGitTransport{inner: inner}
}
func (t *ssrfGitTransport) NewUploadPackSession(ep *gittransport.Endpoint, auth gittransport.AuthMethod) (gittransport.UploadPackSession, error) {
if err := checkEndpointSSRF(ep); err != nil {
return nil, err
}
return t.inner.NewUploadPackSession(ep, auth)
}
func (t *ssrfGitTransport) NewReceivePackSession(ep *gittransport.Endpoint, auth gittransport.AuthMethod) (gittransport.ReceivePackSession, error) {
if err := checkEndpointSSRF(ep); err != nil {
return nil, err
}
return t.inner.NewReceivePackSession(ep, auth)
}
func checkEndpointSSRF(ep *gittransport.Endpoint) error {
port := ep.Port
if port <= 0 {
port = gitDefaultPort
}
rawURL := fmt.Sprintf("git://%s/", net.JoinHostPort(ep.Host, strconv.Itoa(port)))
return ssrf.CheckURL(context.Background(), rawURL)
}

View File

@@ -96,8 +96,4 @@ type GitAuthentication struct {
Password string
Provider GitProvider `json:",omitempty"`
AuthorizationType GitCredentialAuthType `json:",omitempty"`
// Git credentials identifier when the value is not 0
// When the value is 0, Username and Password are set without using saved credential
// This is introduced since 2.15.0
GitCredentialID int `example:"0"`
}

View File

@@ -0,0 +1,55 @@
package sources
import (
portainer "github.com/portainer/portainer/api"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/pkg/fips"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
)
// RepoConfigInput holds the raw payload fields needed to resolve a git RepoConfig.
// Set SourceID to resolve URL/auth from a stored source; otherwise provide the inline fields.
type RepoConfigInput struct {
SourceID portainer.SourceID
ReferenceName string
ConfigFilePath string
RepositoryURL string
TLSSkipVerify bool
RepositoryAuthentication bool
Username string
Password string
Provider gittypes.GitProvider
AuthorizationType gittypes.GitCredentialAuthType
}
// ResolveRepoConfig builds a RepoConfig from either a SourceID or inline URL/auth fields.
func ResolveRepoConfig(tx gitSourceStore, input RepoConfigInput) (gittypes.RepoConfig, *httperror.HandlerError) {
cfg := gittypes.RepoConfig{
ReferenceName: input.ReferenceName,
ConfigFilePath: input.ConfigFilePath,
}
if input.SourceID != 0 {
src, httpErr := ValidateGitSourceAccess(tx, input.SourceID)
if httpErr != nil {
return gittypes.RepoConfig{}, httpErr
}
cfg.URL = src.Git.URL
cfg.Authentication = src.Git.Authentication
cfg.TLSSkipVerify = src.Git.TLSSkipVerify
} else {
cfg.URL = input.RepositoryURL
cfg.TLSSkipVerify = input.TLSSkipVerify
if input.RepositoryAuthentication {
cfg.Authentication = &gittypes.GitAuthentication{
Username: input.Username,
Password: input.Password,
Provider: input.Provider,
AuthorizationType: input.AuthorizationType,
}
}
}
cfg.TLSSkipVerify = cfg.TLSSkipVerify && fips.CanTLSSkipVerify()
return cfg, nil
}

View File

@@ -0,0 +1,70 @@
package sources
import (
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/datastore"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/pkg/fips"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func init() {
fips.InitFIPS(false)
}
func TestResolveRepoConfig_WithSourceID_ReturnsSourceConfig(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, false)
src := &portainer.Source{
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: "https://github.com/org/repo",
TLSSkipVerify: true,
Authentication: &gittypes.GitAuthentication{
Username: "user",
Password: "token",
},
},
}
require.NoError(t, store.Source().Create(src))
cfg, httpErr := ResolveRepoConfig(store, RepoConfigInput{
SourceID: src.ID,
ReferenceName: "refs/heads/main",
ConfigFilePath: "docker-compose.yml",
RepositoryURL: "https://ignored.example.com",
})
require.Nil(t, httpErr)
assert.Equal(t, src.Git.URL, cfg.URL)
assert.Equal(t, src.Git.Authentication, cfg.Authentication)
assert.Equal(t, src.Git.TLSSkipVerify, cfg.TLSSkipVerify)
assert.Equal(t, "refs/heads/main", cfg.ReferenceName)
assert.Equal(t, "docker-compose.yml", cfg.ConfigFilePath)
}
func TestResolveRepoConfig_WithInlineURL_ReturnsInlineConfig(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, false)
cfg, httpErr := ResolveRepoConfig(store, RepoConfigInput{
ReferenceName: "refs/heads/main",
ConfigFilePath: "docker-compose.yml",
RepositoryURL: "https://github.com/org/repo",
TLSSkipVerify: true,
RepositoryAuthentication: true,
Username: "user",
Password: "pass",
})
require.Nil(t, httpErr)
assert.Equal(t, "https://github.com/org/repo", cfg.URL)
assert.True(t, cfg.TLSSkipVerify)
require.NotNil(t, cfg.Authentication)
assert.Equal(t, "user", cfg.Authentication.Username)
assert.Equal(t, "pass", cfg.Authentication.Password)
}

View File

@@ -0,0 +1,38 @@
package sources
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
)
// gitSourceStore is the minimal intersection of CE and EE DataStoreTx that these functions need.
// Both EE and CE DataStoreTx satisfy it, even though they are incompatible as full interface types.
type gitSourceStore interface {
Source() dataservices.SourceService
IsErrObjectNotFound(err error) bool
}
// ValidateGitSourceAccess checks that the given Source exists and is a git Source, and returns it.
// TODO(BE-12905): enforce per-user access policies once Source ownership is introduced.
func ValidateGitSourceAccess(tx gitSourceStore, sourceID portainer.SourceID) (*portainer.Source, *httperror.HandlerError) {
src, err := tx.Source().Read(sourceID)
if err != nil {
if tx.IsErrObjectNotFound(err) {
return nil, httperror.NotFound("Source not found", err)
}
return nil, httperror.InternalServerError("Unable to read source", err)
}
if src.Type != portainer.SourceTypeGit {
return nil, httperror.BadRequest(fmt.Sprintf("source %d is not a git source", sourceID), nil)
}
if src.Git == nil {
return nil, httperror.BadRequest("Source has no git configuration", nil)
}
return src, nil
}

View File

@@ -0,0 +1,49 @@
package sources
import (
"net/http"
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/datastore"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestValidateSourceForStack_ValidGitSource_ReturnsNil(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, false)
src := &portainer.Source{
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{URL: "https://github.com/org/repo"},
}
require.NoError(t, store.Source().Create(src))
_, httpErr := ValidateGitSourceAccess(store, src.ID)
assert.Nil(t, httpErr)
}
func TestValidateSourceForStack_SourceNotFound_Returns404(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, false)
_, httpErr := ValidateGitSourceAccess(store, portainer.SourceID(999))
require.NotNil(t, httpErr)
assert.Equal(t, http.StatusNotFound, httpErr.StatusCode)
}
func TestValidateSourceForStack_NonGitSource_Returns400(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, false)
src := &portainer.Source{
Type: portainer.SourceType(99), // not a git source
}
require.NoError(t, store.Source().Create(src))
_, httpErr := ValidateGitSourceAccess(store, src.ID)
require.NotNil(t, httpErr)
assert.Equal(t, http.StatusBadRequest, httpErr.StatusCode)
}

View File

@@ -9,8 +9,6 @@ import (
"github.com/portainer/portainer/api/http/security"
"github.com/portainer/portainer/api/kubernetes/cli"
"github.com/portainer/portainer/api/set"
"github.com/rs/zerolog/log"
)
// FetchWorkflows returns all GitOps workflows visible to the given user.
@@ -43,69 +41,35 @@ func FetchWorkflows(
// First pass: filter by endpoint/stack-type match and collect workflow IDs.
preFiltered := make([]portainer.Stack, 0, len(stacks))
workflowIDSet := make(map[portainer.WorkflowID]struct{}, len(stacks))
workflowIDSet := make(set.Set[portainer.WorkflowID], len(stacks))
for _, stack := range stacks {
if ep, ok := endpointMap[stack.EndpointID]; ok && !EndpointMatchesStackType(ep, stack.Type) {
continue
}
preFiltered = append(preFiltered, stack)
workflowIDSet[stack.WorkflowID] = struct{}{}
workflowIDSet.Add(stack.WorkflowID)
}
// Batch-load all needed workflows in one scan.
wfs, err := tx.Workflow().ReadAll(func(wf portainer.Workflow) bool {
_, ok := workflowIDSet[wf.ID]
return ok
})
workflowMap, sourceMap, err := LoadWorkflowAndSourceMaps(tx, workflowIDSet)
if err != nil {
return nil, err
}
workflowMap := make(map[portainer.WorkflowID]portainer.Workflow, len(wfs))
var allArtifacts []portainer.ArtifactSources
for _, wf := range wfs {
workflowMap[wf.ID] = wf
allArtifacts = append(allArtifacts, wf.Artifacts...)
}
sourceSet := ArtifactsToSourceSet(allArtifacts...)
// Batch-load all needed sources in one scan.
srcs, err := tx.Source().ReadAll(func(src portainer.Source) bool {
return sourceSet.Contains(src.ID)
})
if err != nil {
return nil, err
}
sourceMap := make(map[portainer.SourceID]portainer.Source, len(srcs))
for _, src := range srcs {
sourceMap[src.ID] = src
}
// Second pass: build filtered list using in-memory lookups.
var filtered []portainer.Stack
for _, stack := range preFiltered {
wf, ok := workflowMap[stack.WorkflowID]
if !ok {
log.Warn().Int("stackID", int(stack.ID)).Msg("workflow record missing for stack, skipping")
continue
}
wf := workflowMap[stack.WorkflowID]
outer:
for _, as := range wf.Artifacts {
if as.Artifact.StackID != stack.ID {
if as.StackID != stack.ID {
continue
}
for _, srcID := range as.SourceIDs {
src, ok := sourceMap[srcID]
if !ok {
log.Warn().Int("stackID", int(stack.ID)).Msg("source record missing for stack, skipping")
break outer
}
for _, f := range as.Files {
src := sourceMap[f.SourceID]
if src.Type == portainer.SourceTypeGit {
gitConfigs[stack.ID] = MergeSourceAndArtifact(&src, &as.Artifact)
gitConfigs[stack.ID] = MergeSourceAndFile(&src, &f)
break outer
}
}
@@ -169,28 +133,27 @@ func FetchSourceStats(
return nil, nil, err
}
workflowIDSet := make(map[portainer.WorkflowID]struct{}, len(allStacks))
workflowIDSet := make(set.Set[portainer.WorkflowID], len(allStacks))
preFiltered := make([]portainer.Stack, 0, len(allStacks))
for _, stack := range allStacks {
if ep, ok := endpointMap[stack.EndpointID]; ok && !EndpointMatchesStackType(ep, stack.Type) {
continue
}
preFiltered = append(preFiltered, stack)
workflowIDSet[stack.WorkflowID] = struct{}{}
workflowIDSet.Add(stack.WorkflowID)
}
wfs, err := tx.Workflow().ReadAll(func(wf portainer.Workflow) bool {
_, ok := workflowIDSet[wf.ID]
return ok
})
wfMap, err := LoadWorkflowMap(tx, workflowIDSet)
if err != nil {
return nil, nil, err
}
wfSources := make(map[portainer.WorkflowID][]portainer.SourceID, len(wfs))
for _, wf := range wfs {
wfSources := make(map[portainer.WorkflowID][]portainer.SourceID, len(wfMap))
for id, wf := range wfMap {
for _, as := range wf.Artifacts {
wfSources[wf.ID] = append(wfSources[wf.ID], as.SourceIDs...)
for _, f := range as.Files {
wfSources[id] = append(wfSources[id], f.SourceID)
}
}
}

View File

@@ -23,12 +23,12 @@ func mustCreateGitWorkflow(t *testing.T, tx dataservices.DataStoreTx, stack *por
cfg := stack.GitConfig
src := &portainer.Source{Type: portainer.SourceTypeGit, GitConfig: cfg}
src := &portainer.Source{Type: portainer.SourceTypeGit, Git: cfg}
require.NoError(t, tx.Source().Create(src))
wf := &portainer.Workflow{Artifacts: []portainer.ArtifactSources{{
Artifact: portainer.Artifact{StackID: stack.ID},
SourceIDs: []portainer.SourceID{src.ID},
wf := &portainer.Workflow{Artifacts: []portainer.Artifact{{
StackID: stack.ID,
Files: []portainer.ArtifactFile{{SourceID: src.ID}},
}}}
require.NoError(t, tx.Workflow().Create(wf))
@@ -228,7 +228,7 @@ func TestFetchSourceStats_TracksWorkflowCountAndEndpoints(t *testing.T) {
srcID = src.ID
for i := 1; i <= 2; i++ {
wf := &portainer.Workflow{Artifacts: []portainer.ArtifactSources{{SourceIDs: []portainer.SourceID{srcID}}}}
wf := &portainer.Workflow{Artifacts: []portainer.Artifact{{Files: []portainer.ArtifactFile{{SourceID: srcID}}}}}
require.NoError(t, tx.Workflow().Create(wf))
require.NoError(t, tx.Stack().Create(&portainer.Stack{
ID: portainer.StackID(i),

View File

@@ -16,6 +16,8 @@ import (
"github.com/portainer/portainer/api/set"
"github.com/portainer/portainer/api/slicesx"
"github.com/portainer/portainer/api/stacks/stackutils"
"github.com/rs/zerolog/log"
)
func EndpointMatchesStackType(ep portainer.Endpoint, stackType portainer.StackType) bool {
@@ -115,7 +117,8 @@ func buildEndpointAccessMap(k8sFactory *cli.ClientFactory, sc *security.Restrict
access, err := resolveKubeAccess(k8sFactory, sc, &ep)
if err != nil {
return nil, err
log.Warn().Err(err).Str("context", "buildEndpointAccessMap").Int("endpoint_id", int(epID)).Msg("Failed to resolve kube access for endpoint, skipping")
continue
}
result[epID] = access
@@ -148,7 +151,8 @@ func filterK8SStacks(items []portainer.Stack, endpointMap map[portainer.Endpoint
kcl, err := k8sFactory.GetPrivilegedKubeClient(&ep)
if err != nil {
return nil, err
log.Warn().Err(err).Str("context", "filterK8SStacks").Int("endpoint_id", int(envID)).Msg("Failed to get kube client for endpoint, skipping")
continue
}
access := accessMap[envID]
@@ -157,7 +161,8 @@ func filterK8SStacks(items []portainer.Stack, endpointMap map[portainer.Endpoint
apps, err := kcl.GetApplications("", "")
if err != nil {
return nil, err
log.Warn().Err(err).Str("context", "filterK8SStacks").Int("endpoint_id", int(envID)).Msg("Failed to get kube applications for endpoint, skipping")
continue
}
for _, s := range stacks {

View File

@@ -1,6 +1,8 @@
package workflows
import (
"slices"
portainer "github.com/portainer/portainer/api"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/set"
@@ -61,9 +63,9 @@ func MapEdgeStackToWorkflow(es portainer.EdgeStack, gitConfig *gittypes.RepoConf
}
func StackLastSyncDate(s portainer.Stack) int64 {
for i := len(s.DeploymentStatus) - 1; i >= 0; i-- {
if s.DeploymentStatus[i].Status == portainer.StackStatusActive {
return s.DeploymentStatus[i].Time
for _, ds := range slices.Backward(s.DeploymentStatus) {
if ds.Status == portainer.StackStatusActive {
return ds.Time
}
}
return 0
@@ -84,9 +86,9 @@ func edgeStackLastSyncDate(statuses []portainer.EdgeStackStatusForEnv) int64 {
}
func endpointLastSyncDate(epStatus portainer.EdgeStackStatusForEnv) int64 {
for i := len(epStatus.Status) - 1; i >= 0; i-- {
if isEdgeStackHealthyStatus(epStatus.Status[i].Type) {
return epStatus.Status[i].Time
for _, s := range slices.Backward(epStatus.Status) {
if isEdgeStackHealthyStatus(s.Type) {
return s.Time
}
}
return 0
@@ -115,18 +117,6 @@ func isEdgeStackHealthyStatus(t portainer.EdgeStackStatusType) bool {
return false
}
// ArtifactsToSourceSet returns the set of all SourceIDs referenced by the given artifact-source mappings
func ArtifactsToSourceSet(artifacts ...portainer.ArtifactSources) set.Set[portainer.SourceID] {
s := make(set.Set[portainer.SourceID])
for _, a := range artifacts {
for _, sid := range a.SourceIDs {
s.Add(sid)
}
}
return s
}
func resolveEdgeGroupEndpoints(groups []portainer.EdgeGroupID, groupEndpoints map[portainer.EdgeGroupID][]portainer.EndpointID) []portainer.EndpointID {
seen := set.Set[portainer.EndpointID]{}
for _, gid := range groups {

View File

@@ -6,19 +6,21 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/set"
)
// gitSourceStore is the minimal intersection of CE and EE DataStoreTx that these functions need.
// Both EE and CE DataStoreTx satisfy it, even though they are incompatible as full interface types.
type gitSourceStore interface {
Workflow() dataservices.WorkflowService
Source() dataservices.SourceService
}
// GitSourceAndArtifactForStack returns the git Source and the Artifact matching stackID
// GitSourceAndArtifactForStack returns the git Source and the ArtifactFile matching stackID
// from the workflow identified by workflowID.
// Source carries the shared fields (URL, auth, TLS); Artifact carries the stack-specific fields (ref, path, hash).
// Source carries the shared fields (URL, auth, TLS); ArtifactFile carries the file-specific fields (ref, path, hash).
// Returns nil, nil, nil when workflowID is 0 or no matching entry is found.
func GitSourceAndArtifactForStack(tx gitSourceStore, workflowID portainer.WorkflowID, stackID portainer.StackID) (*portainer.Source, *portainer.Artifact, error) {
func GitSourceAndArtifactForStack(tx gitSourceStore, workflowID portainer.WorkflowID, stackID portainer.StackID) (*portainer.Source, *portainer.ArtifactFile, error) {
if workflowID == 0 {
return nil, nil, nil
}
@@ -28,19 +30,24 @@ func GitSourceAndArtifactForStack(tx gitSourceStore, workflowID portainer.Workfl
return nil, nil, err
}
sourceMap, err := loadWorkflowSources(tx, wf)
if err != nil {
return nil, nil, err
}
for i, as := range wf.Artifacts {
if as.Artifact.StackID != stackID {
if as.StackID != stackID {
continue
}
for _, srcID := range as.SourceIDs {
src, err := tx.Source().Read(srcID)
if err != nil {
return nil, nil, err
for j, file := range as.Files {
src, ok := sourceMap[file.SourceID]
if !ok {
continue
}
if src.Type == portainer.SourceTypeGit {
return src, &wf.Artifacts[i].Artifact, nil
return &src, &wf.Artifacts[i].Files[j], nil
}
}
}
@@ -48,9 +55,9 @@ func GitSourceAndArtifactForStack(tx gitSourceStore, workflowID portainer.Workfl
return nil, nil, nil
}
// GitSourceAndArtifactForEdgeStack returns the git Source and the Artifact matching edgeStackID.
// GitSourceAndArtifactForEdgeStack returns the git Source and the ArtifactFile matching edgeStackID.
// Returns nil, nil, nil when workflowID is 0 or no matching entry is found.
func GitSourceAndArtifactForEdgeStack(tx gitSourceStore, workflowID portainer.WorkflowID, edgeStackID portainer.EdgeStackID) (*portainer.Source, *portainer.Artifact, error) {
func GitSourceAndArtifactForEdgeStack(tx gitSourceStore, workflowID portainer.WorkflowID, edgeStackID portainer.EdgeStackID) (*portainer.Source, *portainer.ArtifactFile, error) {
if workflowID == 0 {
return nil, nil, nil
}
@@ -60,19 +67,24 @@ func GitSourceAndArtifactForEdgeStack(tx gitSourceStore, workflowID portainer.Wo
return nil, nil, err
}
sourceMap, err := loadWorkflowSources(tx, wf)
if err != nil {
return nil, nil, err
}
for i, as := range wf.Artifacts {
if as.Artifact.EdgeStackID != edgeStackID {
if as.EdgeStackID != edgeStackID {
continue
}
for _, srcID := range as.SourceIDs {
src, err := tx.Source().Read(srcID)
if err != nil {
return nil, nil, err
for j, file := range as.Files {
src, ok := sourceMap[file.SourceID]
if !ok {
continue
}
if src.Type == portainer.SourceTypeGit {
return src, &wf.Artifacts[i].Artifact, nil
return &src, &wf.Artifacts[i].Files[j], nil
}
}
}
@@ -80,60 +92,74 @@ func GitSourceAndArtifactForEdgeStack(tx gitSourceStore, workflowID portainer.Wo
return nil, nil, nil
}
// MergeSourceAndArtifact builds a RepoConfig by combining shared fields from src (URL, auth, TLS)
// with stack-specific fields from artifact (ref, path, hash).
func MergeSourceAndArtifact(src *portainer.Source, artifact *portainer.Artifact) *gittypes.RepoConfig {
if src == nil || src.GitConfig == nil {
// MergeSourceAndFile builds a RepoConfig by combining shared fields from src (URL, auth, TLS)
// with file-specific fields from file (ref, path, hash).
func MergeSourceAndFile(src *portainer.Source, file *portainer.ArtifactFile) *gittypes.RepoConfig {
if src == nil || src.Git == nil {
return nil
}
cfg := &gittypes.RepoConfig{
URL: src.GitConfig.URL,
Authentication: src.GitConfig.Authentication,
TLSSkipVerify: src.GitConfig.TLSSkipVerify,
URL: src.Git.URL,
Authentication: src.Git.Authentication,
TLSSkipVerify: src.Git.TLSSkipVerify,
}
if artifact != nil {
cfg.ReferenceName = artifact.ReferenceName
cfg.ConfigFilePath = artifact.ConfigFilePath
cfg.ConfigHash = artifact.ConfigHash
if file != nil {
cfg.ReferenceName = file.Ref
cfg.ConfigFilePath = file.Path
cfg.ConfigHash = file.Hash
}
return cfg
}
// UpdateArtifactForStack finds the Artifact matching stackID in the workflow and applies fn to it,
// then persists the updated Workflow. A no-op if no matching Artifact is found.
func UpdateArtifactForStack(tx gitSourceStore, workflowID portainer.WorkflowID, stackID portainer.StackID, fn func(*portainer.Artifact)) error {
// UpdateArtifactFileForStack finds the ArtifactFile matching stackID and sourceID in the workflow
// and applies fn to it, then persists the updated Workflow.
// A no-op if no matching artifact or file is found.
func UpdateArtifactFileForStack(tx gitSourceStore, workflowID portainer.WorkflowID, stackID portainer.StackID, sourceID portainer.SourceID, fn func(*portainer.ArtifactFile)) error {
wf, err := tx.Workflow().Read(workflowID)
if err != nil {
return err
}
for i, as := range wf.Artifacts {
if as.Artifact.StackID == stackID {
fn(&wf.Artifacts[i].Artifact)
if as.StackID != stackID {
continue
}
return tx.Workflow().Update(workflowID, wf)
for j, file := range as.Files {
if file.SourceID == sourceID {
fn(&wf.Artifacts[i].Files[j])
return tx.Workflow().Update(workflowID, wf)
}
}
}
return nil
}
// UpdateArtifactForEdgeStack finds the Artifact matching edgeStackID in the workflow and applies fn to it,
// then persists the updated Workflow. A no-op if no matching Artifact is found.
func UpdateArtifactForEdgeStack(tx gitSourceStore, workflowID portainer.WorkflowID, edgeStackID portainer.EdgeStackID, fn func(*portainer.Artifact)) error {
// UpdateArtifactFileForEdgeStack finds the ArtifactFile matching edgeStackID and sourceID in the workflow
// and applies fn to it, then persists the updated Workflow.
// A no-op if no matching artifact or file is found.
func UpdateArtifactFileForEdgeStack(tx gitSourceStore, workflowID portainer.WorkflowID, edgeStackID portainer.EdgeStackID, sourceID portainer.SourceID, fn func(*portainer.ArtifactFile)) error {
wf, err := tx.Workflow().Read(workflowID)
if err != nil {
return err
}
for i, as := range wf.Artifacts {
if as.Artifact.EdgeStackID == edgeStackID {
fn(&wf.Artifacts[i].Artifact)
if as.EdgeStackID != edgeStackID {
continue
}
return tx.Workflow().Update(workflowID, wf)
for j, file := range as.Files {
if file.SourceID == sourceID {
fn(&wf.Artifacts[i].Files[j])
return tx.Workflow().Update(workflowID, wf)
}
}
}
@@ -144,13 +170,13 @@ func UpdateArtifactForEdgeStack(tx gitSourceStore, workflowID portainer.Workflow
// or creates a new one. Only URL, authentication, and TLSSkipVerify are stored on the Source;
// per-stack fields (ReferenceName, ConfigFilePath, ConfigHash) belong in the Artifact.
func FindOrCreateGitSource(tx gitSourceStore, src *portainer.Source) (*portainer.Source, error) {
src.GitConfig.URL = gittypes.SanitizeURL(src.GitConfig.URL)
src.Git.URL = gittypes.SanitizeURL(src.Git.URL)
existing, err := tx.Source().ReadAll(func(s portainer.Source) bool {
return s.Type == portainer.SourceTypeGit &&
s.GitConfig != nil &&
s.GitConfig.URL == src.GitConfig.URL &&
gitAuthMatches(s.GitConfig.Authentication, src.GitConfig.Authentication)
s.Git != nil &&
s.Git.URL == src.Git.URL &&
gitAuthMatches(s.Git.Authentication, src.Git.Authentication)
})
if err != nil {
return nil, err
@@ -163,10 +189,10 @@ func FindOrCreateGitSource(tx gitSourceStore, src *portainer.Source) (*portainer
toCreate := &portainer.Source{
Name: src.Name,
Type: portainer.SourceTypeGit,
GitConfig: &gittypes.RepoConfig{
URL: src.GitConfig.URL,
Authentication: src.GitConfig.Authentication,
TLSSkipVerify: src.GitConfig.TLSSkipVerify,
Git: &gittypes.RepoConfig{
URL: src.Git.URL,
Authentication: src.Git.Authentication,
TLSSkipVerify: src.Git.TLSSkipVerify,
},
}
@@ -186,17 +212,17 @@ func SaveWorkflowGitConfig(tx gitSourceStore, workflowID portainer.WorkflowID, m
return fmt.Errorf("failed to read source: %w", err)
}
if src.GitConfig == nil {
if src.Git == nil {
return fmt.Errorf("source %d has no git configuration", oldSourceID)
}
newSourceID := oldSourceID
if cfg.URL != src.GitConfig.URL {
if cfg.URL != src.Git.URL {
newSrc, err := FindOrCreateGitSource(tx, &portainer.Source{
Name: gittypes.RepoName(cfg.URL),
Type: portainer.SourceTypeGit,
GitConfig: cfg,
Name: gittypes.RepoName(cfg.URL),
Type: portainer.SourceTypeGit,
Git: cfg,
})
if err != nil {
return fmt.Errorf("failed to find or create source: %w", err)
@@ -204,34 +230,49 @@ func SaveWorkflowGitConfig(tx gitSourceStore, workflowID portainer.WorkflowID, m
newSourceID = newSrc.ID
} else {
src.GitConfig.Authentication = cfg.Authentication
src.GitConfig.TLSSkipVerify = cfg.TLSSkipVerify
src.Git.Authentication = cfg.Authentication
src.Git.TLSSkipVerify = cfg.TLSSkipVerify
if err := tx.Source().Update(src.ID, src); err != nil {
return fmt.Errorf("failed to update source: %w", err)
}
}
return SaveWorkflowArtifact(tx, workflowID, matchArtifact, oldSourceID, portainer.ArtifactFile{
SourceID: newSourceID,
Ref: cfg.ReferenceName,
Path: cfg.ConfigFilePath,
Hash: cfg.ConfigHash,
})
}
// SaveWorkflowArtifact replaces the ArtifactFile referencing oldSourceID on the Artifact matched by
// matchArtifact with update (its SourceID may repoint the Artifact to a different Source). It does not
// modify any Source's git config — the caller is responsible for ensuring update.SourceID
// references a valid existing Source.
func SaveWorkflowArtifact(tx gitSourceStore, workflowID portainer.WorkflowID, matchArtifact func(portainer.Artifact) bool, oldSourceID portainer.SourceID, update portainer.ArtifactFile) error {
wf, err := tx.Workflow().Read(workflowID)
if err != nil {
return fmt.Errorf("failed to read workflow: %w", err)
}
for i, as := range wf.Artifacts {
if !matchArtifact(as.Artifact) {
if !matchArtifact(as) {
continue
}
wf.Artifacts[i].Artifact.ReferenceName = cfg.ReferenceName
wf.Artifacts[i].Artifact.ConfigFilePath = cfg.ConfigFilePath
wf.Artifacts[i].Artifact.ConfigHash = cfg.ConfigHash
if newSourceID != oldSourceID {
for j, sID := range as.SourceIDs {
if sID == oldSourceID {
wf.Artifacts[i].SourceIDs[j] = newSourceID
}
for j, file := range as.Files {
if file.SourceID != oldSourceID {
continue
}
f := &wf.Artifacts[i].Files[j]
f.SourceID = update.SourceID
f.Ref = update.Ref
f.Path = update.Path
f.Hash = update.Hash
break
}
break
@@ -240,6 +281,73 @@ func SaveWorkflowGitConfig(tx gitSourceStore, workflowID portainer.WorkflowID, m
return tx.Workflow().Update(workflowID, wf)
}
// LoadWorkflowMap fetches workflows by their IDs and returns them keyed by ID.
func LoadWorkflowMap(tx gitSourceStore, ids set.Set[portainer.WorkflowID]) (map[portainer.WorkflowID]portainer.Workflow, error) {
result := make(map[portainer.WorkflowID]portainer.Workflow, len(ids))
for id := range ids {
wf, err := tx.Workflow().Read(id)
if err != nil {
return nil, err
}
result[id] = *wf
}
return result, nil
}
// LoadWorkflowAndSourceMaps fetches workflows by their IDs and the sources they reference,
// collecting source IDs in a single pass over the workflows.
func LoadWorkflowAndSourceMaps(tx gitSourceStore, ids set.Set[portainer.WorkflowID]) (map[portainer.WorkflowID]portainer.Workflow, map[portainer.SourceID]portainer.Source, error) {
wfMap := make(map[portainer.WorkflowID]portainer.Workflow, len(ids))
sourceIDs := make(set.Set[portainer.SourceID])
for id := range ids {
wf, err := tx.Workflow().Read(id)
if err != nil {
return nil, nil, err
}
wfMap[id] = *wf
for _, as := range wf.Artifacts {
for _, f := range as.Files {
sourceIDs.Add(f.SourceID)
}
}
}
srcMap, err := LoadSourceMap(tx, sourceIDs)
if err != nil {
return nil, nil, err
}
return wfMap, srcMap, nil
}
// loadWorkflowSources collects all unique SourceIDs referenced by wf and returns them as a map.
// This avoids reading the same Source record more than once when files share a SourceID.
func loadWorkflowSources(tx gitSourceStore, wf *portainer.Workflow) (map[portainer.SourceID]portainer.Source, error) {
ids := make(set.Set[portainer.SourceID])
for _, as := range wf.Artifacts {
for _, f := range as.Files {
ids.Add(f.SourceID)
}
}
return LoadSourceMap(tx, ids)
}
// LoadSourceMap fetches sources by their IDs and returns them keyed by ID.
func LoadSourceMap(tx gitSourceStore, ids set.Set[portainer.SourceID]) (map[portainer.SourceID]portainer.Source, error) {
result := make(map[portainer.SourceID]portainer.Source, len(ids))
for id := range ids {
src, err := tx.Source().Read(id)
if err != nil {
return nil, err
}
result[id] = *src
}
return result, nil
}
func gitAuthMatches(a, b *gittypes.GitAuthentication) bool {
if a == nil && b == nil {
return true
@@ -249,24 +357,29 @@ func gitAuthMatches(a, b *gittypes.GitAuthentication) bool {
return false
}
return a.Username == b.Username && a.Password == b.Password && a.GitCredentialID == b.GitCredentialID
return a.Username == b.Username && a.Password == b.Password
}
// ValidateUniqueSourceURL validates there are no other sources with the same URL
func ValidateUniqueSourceURL(tx gitSourceStore, url string, sourceID portainer.SourceID) (bool, error) {
// ValidateUniqueSource validates there are no other sources with the same URL and credentials.
// Pass empty strings for username and password when the source has no authentication.
func ValidateUniqueSource(tx gitSourceStore, url, username, password string, sourceID portainer.SourceID) (bool, error) {
normalizedURL, err := gittypes.NormalizeURL(gittypes.SanitizeURL(url))
if err != nil {
return false, err
}
existing, err := tx.Source().ReadAll(func(s portainer.Source) bool {
if s.ID == sourceID || s.Type != portainer.SourceTypeGit || s.GitConfig == nil {
if s.ID == sourceID || s.Type != portainer.SourceTypeGit || s.Git == nil {
return false
}
normalized, err := gittypes.NormalizeURL(gittypes.SanitizeURL(s.GitConfig.URL))
normalized, err := gittypes.NormalizeURL(gittypes.SanitizeURL(s.Git.URL))
if err != nil || normalized != normalizedURL {
return false
}
return err == nil && normalized == normalizedURL
existingUsername, existingPassword := gitAuthCredentials(s.Git.Authentication)
return existingUsername == username && existingPassword == password
})
if err != nil {
@@ -275,3 +388,10 @@ func ValidateUniqueSourceURL(tx gitSourceStore, url string, sourceID portainer.S
return len(existing) == 0, nil
}
func gitAuthCredentials(auth *gittypes.GitAuthentication) (username, password string) {
if auth == nil {
return "", ""
}
return auth.Username, auth.Password
}

File diff suppressed because it is too large Load Diff

View File

@@ -11,6 +11,7 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/crypto"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
"github.com/rs/zerolog/log"
"github.com/segmentio/encoding/json"
@@ -114,18 +115,19 @@ func Get(url string, timeout int) ([]byte, error) {
// using the specified host and optional TLS configuration.
// It uses a new Http.Client for each operation.
func ExecutePingOperation(host string, tlsConfiguration portainer.TLSConfiguration) (bool, error) {
transport := &http.Transport{}
scheme := "http"
var transport *http.Transport
if tlsConfiguration.TLS {
tlsConfig, err := crypto.CreateTLSConfigurationFromDisk(tlsConfiguration)
if err != nil {
return false, err
}
transport.TLSClientConfig = tlsConfig
scheme = "https"
transport = ssrf.NewTransport(tlsConfig)
} else {
transport = ssrf.NewTransport(nil)
}
client := &http.Client{

View File

@@ -22,6 +22,7 @@ type Handler struct {
filestorePath string
shutdownTrigger context.CancelFunc
adminMonitor *adminmonitor.Monitor
SetupToken string
}
// NewHandler creates an new instance of backup handler

View File

@@ -7,6 +7,7 @@ import (
"github.com/pkg/errors"
operations "github.com/portainer/portainer/api/backup"
"github.com/portainer/portainer/api/http/security/setuptoken"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
)
@@ -20,15 +21,21 @@ type restorePayload struct {
// @id Restore
// @summary Triggers a system restore using provided backup file
// @description Triggers a system restore using provided backup file
// @description **Access policy**: public
// @description **Access policy**: public (requires the X-Setup-Token header on an uninitialized instance unless --no-setup-token is set)
// @tags backup
// @accept json
// @param X-Setup-Token header string false "Setup token (required when instance is uninitialized and --no-setup-token is not set)"
// @param restorePayload body restorePayload true "Restore request payload"
// @success 200 "Success"
// @failure 400 "Invalid request"
// @failure 403 "Access denied - invalid or missing setup token"
// @failure 500 "Server error"
// @router /restore [post]
func (h *Handler) restore(w http.ResponseWriter, r *http.Request) *httperror.HandlerError {
if err := setuptoken.Validate(r, h.SetupToken); err != nil {
return err
}
initialized, err := h.adminMonitor.WasInitialized()
if err != nil {
return httperror.InternalServerError("Failed to check system initialization", err)

View File

@@ -14,6 +14,7 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/adminmonitor"
"github.com/portainer/portainer/api/http/offlinegate"
"github.com/portainer/portainer/api/http/security/setuptoken"
"github.com/portainer/portainer/api/internal/testhelpers"
"github.com/stretchr/testify/assert"
@@ -126,6 +127,45 @@ func backup(t *testing.T, h *Handler, password string) []byte {
return archive
}
func Test_restore_setupTokenGate(t *testing.T) {
t.Parallel()
datastore := testhelpers.NewDatastore(
testhelpers.WithUsers([]portainer.User{}),
testhelpers.WithEdgeJobs([]portainer.EdgeJob{}),
)
adminMonitor := adminmonitor.New(time.Hour, datastore)
h := NewHandler(
testhelpers.NewTestRequestBouncer(),
datastore,
offlinegate.NewOfflineGate(),
prepareFilestorePath(t),
func() {},
adminMonitor,
)
h.SetupToken = "secret-token"
t.Run("403 without token header", func(t *testing.T) {
err := h.restore(httptest.NewRecorder(), prepareMultipartRequest(t, "", []byte("x")))
require.Error(t, err)
assert.Equal(t, http.StatusForbidden, err.StatusCode)
})
t.Run("403 with wrong token", func(t *testing.T) {
r := prepareMultipartRequest(t, "", []byte("x"))
r.Header.Set(setuptoken.HeaderName, "wrong")
err := h.restore(httptest.NewRecorder(), r)
require.Error(t, err)
assert.Equal(t, http.StatusForbidden, err.StatusCode)
})
t.Run("passes gate with correct token", func(t *testing.T) {
archive := backup(t, h, "")
r := prepareMultipartRequest(t, "", archive)
r.Header.Set(setuptoken.HeaderName, "secret-token")
require.Nil(t, h.restore(httptest.NewRecorder(), r))
})
}
func prepareMultipartRequest(t *testing.T, password string, file []byte) *http.Request {
var body bytes.Buffer

View File

@@ -5,18 +5,21 @@ import (
"errors"
"net/http"
"os"
"regexp"
"strconv"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/filesystem"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/gitops/sources"
"github.com/portainer/portainer/api/gitops/workflows"
"github.com/portainer/portainer/api/http/security"
"github.com/portainer/portainer/api/internal/authorization"
"github.com/portainer/portainer/api/stacks/stackutils"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
"github.com/portainer/portainer/pkg/validate"
"github.com/rs/zerolog/log"
@@ -41,30 +44,39 @@ func (handler *Handler) customTemplateCreate(w http.ResponseWriter, r *http.Requ
customTemplate.CreatedByUserID = tokenData.ID
customTemplates, err := handler.DataStore.CustomTemplate().ReadAll()
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
return createCustomTemplateTx(tx, customTemplate, tokenData.ID)
})
return response.TxResponse(w, customTemplate, err)
}
func createCustomTemplateTx(tx dataservices.DataStoreTx, customTemplate *portainer.CustomTemplate, userID portainer.UserID) error {
existingTemplates, err := tx.CustomTemplate().ReadAll()
if err != nil {
return httperror.InternalServerError("Unable to retrieve custom templates from the database", err)
}
for _, existingTemplate := range customTemplates {
if existingTemplate.Title == customTemplate.Title {
for _, existing := range existingTemplates {
if existing.Title == customTemplate.Title {
return httperror.InternalServerError("Template name must be unique", errors.New("Template name must be unique"))
}
}
if err := handler.DataStore.CustomTemplate().Create(customTemplate); err != nil {
if err := tx.CustomTemplate().Create(customTemplate); err != nil {
return httperror.InternalServerError("Unable to create custom template", err)
}
resourceControl := authorization.NewPrivateResourceControl(strconv.Itoa(int(customTemplate.ID)), portainer.CustomTemplateResourceControl, tokenData.ID)
resourceControl := authorization.NewPrivateResourceControl(strconv.Itoa(int(customTemplate.ID)), portainer.CustomTemplateResourceControl, userID)
if err := handler.DataStore.ResourceControl().Create(resourceControl); err != nil {
if err := tx.ResourceControl().Create(resourceControl); err != nil {
return httperror.InternalServerError("Unable to persist resource control inside the database", err)
}
customTemplate.ResourceControl = resourceControl
populateGitConfig(tx, customTemplate)
return response.JSON(w, customTemplate)
return nil
}
func (handler *Handler) createCustomTemplate(method string, r *http.Request) (*portainer.CustomTemplate, error) {
@@ -122,19 +134,11 @@ func (payload *customTemplateFromFileContentPayload) Validate(r *http.Request) e
if payload.Type != portainer.KubernetesStack && payload.Type != portainer.DockerSwarmStack && payload.Type != portainer.DockerComposeStack {
return errors.New("Invalid custom template type")
}
if !isValidNote(payload.Note) {
if !IsValidNote(payload.Note) {
return errors.New("Invalid note. <img> tag is not supported")
}
return validateVariablesDefinitions(payload.Variables)
}
func isValidNote(note string) bool {
if len(note) == 0 {
return true
}
match, _ := regexp.MatchString("<img", note)
return !match
return ValidateVariablesDefinitions(payload.Variables)
}
// @id CustomTemplateCreateString
@@ -200,21 +204,24 @@ type customTemplateFromGitRepositoryPayload struct {
// * 3 - kubernetes
Type portainer.StackType `example:"1" enums:"1,2" validate:"required"`
// URL of a Git repository hosting the Stack file
RepositoryURL string `example:"https://github.com/openfaas/faas" validate:"required"`
// SourceID references an existing Source for git credentials/URL.
// When set, the inline URL and authentication fields are ignored.
SourceID portainer.SourceID `example:"1" validate:"required"`
// Deprecated: use SourceID instead. URL of a Git repository hosting the Stack file.
RepositoryURL string `example:"https://github.com/openfaas/faas"`
// Reference name of a Git repository hosting the Stack file
RepositoryReferenceName string `example:"refs/heads/master"`
// Use basic authentication to clone the Git repository
// Deprecated: use SourceID instead. Use basic authentication to clone the Git repository.
RepositoryAuthentication bool `example:"true"`
// Username used in basic authentication. Required when RepositoryAuthentication is true.
// Deprecated: use SourceID instead. Username used in basic authentication. Required when RepositoryAuthentication is true.
RepositoryUsername string `example:"myGitUsername"`
// Password used in basic authentication. Required when RepositoryAuthentication is true.
// Deprecated: use SourceID instead. Password used in basic authentication. Required when RepositoryAuthentication is true.
RepositoryPassword string `example:"myGitPassword"`
// Path to the Stack file inside the Git repository
ComposeFilePathInRepository string `example:"docker-compose.yml" default:"docker-compose.yml"`
// Definitions of variables in the stack file
Variables []portainer.CustomTemplateVariableDefinition
// TLSSkipVerify skips SSL verification when cloning the Git repository
// Deprecated: use SourceID instead. TLSSkipVerify skips SSL verification when cloning the Git repository.
TLSSkipVerify bool `example:"false"`
// IsComposeFormat indicates if the Kubernetes template is created from a Docker Compose file
IsComposeFormat bool `example:"false"`
@@ -229,11 +236,13 @@ func (payload *customTemplateFromGitRepositoryPayload) Validate(r *http.Request)
if len(payload.Description) == 0 {
return errors.New("Invalid custom template description")
}
if len(payload.RepositoryURL) == 0 || !validate.IsURL(payload.RepositoryURL) {
return errors.New("Invalid repository URL. Must correspond to a valid URL format")
}
if payload.RepositoryAuthentication && (len(payload.RepositoryUsername) == 0 || len(payload.RepositoryPassword) == 0) {
return errors.New("Invalid repository credentials. Username and password must be specified when authentication is enabled")
if payload.SourceID == 0 {
if len(payload.RepositoryURL) == 0 || !validate.IsURL(payload.RepositoryURL) {
return errors.New("Invalid repository URL. Must correspond to a valid URL format")
}
if payload.RepositoryAuthentication && (len(payload.RepositoryUsername) == 0 || len(payload.RepositoryPassword) == 0) {
return errors.New("Invalid repository credentials. Username and password must be specified when authentication is enabled")
}
}
if len(payload.ComposeFilePathInRepository) == 0 {
payload.ComposeFilePathInRepository = filesystem.ComposeFileDefaultName
@@ -246,11 +255,11 @@ func (payload *customTemplateFromGitRepositoryPayload) Validate(r *http.Request)
if payload.Type != portainer.DockerSwarmStack && payload.Type != portainer.DockerComposeStack && payload.Type != portainer.KubernetesStack {
return errors.New("Invalid custom template type")
}
if !isValidNote(payload.Note) {
if !IsValidNote(payload.Note) {
return errors.New("Invalid note. <img> tag is not supported")
}
return validateVariablesDefinitions(payload.Variables)
return ValidateVariablesDefinitions(payload.Variables)
}
// @id CustomTemplateCreateRepository
@@ -293,28 +302,54 @@ func (handler *Handler) createCustomTemplateFromGitRepository(r *http.Request) (
projectPath := getProjectPath()
customTemplate.ProjectPath = projectPath
gitConfig := &gittypes.RepoConfig{
URL: payload.RepositoryURL,
ReferenceName: payload.RepositoryReferenceName,
ConfigFilePath: payload.ComposeFilePathInRepository,
TLSSkipVerify: payload.TLSSkipVerify,
gitConfig, httpErr := sources.ResolveRepoConfig(handler.DataStore, sources.RepoConfigInput{
SourceID: payload.SourceID,
ReferenceName: payload.RepositoryReferenceName,
ConfigFilePath: payload.ComposeFilePathInRepository,
RepositoryURL: payload.RepositoryURL,
TLSSkipVerify: payload.TLSSkipVerify,
RepositoryAuthentication: payload.RepositoryAuthentication,
Username: payload.RepositoryUsername,
Password: payload.RepositoryPassword,
})
if httpErr != nil {
return nil, httpErr
}
if payload.RepositoryAuthentication {
gitConfig.Authentication = &gittypes.GitAuthentication{
Username: payload.RepositoryUsername,
Password: payload.RepositoryPassword,
}
if err := ssrf.CheckURL(r.Context(), gitConfig.URL); err != nil {
return nil, err
}
commitHash, err := stackutils.DownloadGitRepository(context.TODO(), *gitConfig, handler.GitService, getProjectPath)
commitHash, err := stackutils.DownloadGitRepository(context.TODO(), gitConfig, handler.GitService, getProjectPath)
if err != nil {
return nil, err
}
gitConfig.ConfigHash = commitHash
customTemplate.GitConfig = gitConfig
sourceID := payload.SourceID
if sourceID == 0 {
src, err := workflows.FindOrCreateGitSource(handler.DataStore, &portainer.Source{
Name: gittypes.RepoName(gitConfig.URL),
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: gitConfig.URL,
Authentication: gitConfig.Authentication,
TLSSkipVerify: gitConfig.TLSSkipVerify,
},
})
if err != nil {
return nil, err
}
sourceID = src.ID
}
customTemplate.Artifact = &portainer.Artifact{
Files: []portainer.ArtifactFile{{
SourceID: sourceID,
Path: gitConfig.ConfigFilePath,
Ref: gitConfig.ReferenceName,
Hash: commitHash,
}},
}
isValidProject := true
defer func() {
if !isValidProject {
@@ -390,7 +425,7 @@ func (payload *customTemplateFromFileUploadPayload) Validate(r *http.Request) er
payload.Logo = logo
note, _ := request.RetrieveMultiPartFormValue(r, "Note", true)
if !isValidNote(note) {
if !IsValidNote(note) {
return errors.New("Invalid note. <img> tag is not supported")
}
payload.Note = note
@@ -422,7 +457,7 @@ func (payload *customTemplateFromFileUploadPayload) Validate(r *http.Request) er
if err := json.Unmarshal([]byte(varsString), &payload.Variables); err != nil {
return errors.New("Invalid variables. Ensure that the variables are valid JSON")
}
if err := validateVariablesDefinitions(payload.Variables); err != nil {
if err := ValidateVariablesDefinitions(payload.Variables); err != nil {
return err
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -51,6 +51,7 @@ func (handler *Handler) customTemplateDelete(w http.ResponseWriter, r *http.Requ
return httperror.InternalServerError("Unable to retrieve a resource control associated to the custom template", err)
}
customTemplate.ResourceControl = resourceControl
access := userCanEditTemplate(customTemplate, securityContext)
if !access {
return httperror.Forbidden("Access denied to resource", httperrors.ErrResourceAccessDenied)

View File

@@ -0,0 +1,296 @@
package customtemplates
import (
"net/http"
"net/http/httptest"
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/http/security"
"github.com/gorilla/mux"
"github.com/stretchr/testify/require"
)
func TestCustomTemplateDelete_NotFound(t *testing.T) {
t.Parallel()
handler, _, _ := newTestHandler(t)
r := httptest.NewRequest(http.MethodDelete, "/custom_templates/99", nil)
r = mux.SetURLVars(r, map[string]string{"id": "99"})
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true}))
rr := httptest.NewRecorder()
herr := handler.customTemplateDelete(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusNotFound, herr.StatusCode)
}
func TestCustomTemplateDelete_Forbidden(t *testing.T) {
t.Parallel()
handler, store, _ := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
err := tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
CreatedByUserID: 1,
})
require.NoError(t, err)
return nil
})
require.NoError(t, err)
// User 2 did not create this template and is not an admin
r := httptest.NewRequest(http.MethodDelete, "/custom_templates/1", nil)
r = mux.SetURLVars(r, map[string]string{"id": "1"})
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 2}))
rr := httptest.NewRecorder()
herr := handler.customTemplateDelete(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusForbidden, herr.StatusCode)
}
func TestCustomTemplateDelete_CreatorDeniedWhenAdminOnly(t *testing.T) {
t.Parallel()
handler, store, _ := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
err := tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
CreatedByUserID: 2,
})
require.NoError(t, err)
err = tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 1,
ResourceID: "1",
Type: portainer.CustomTemplateResourceControl,
AdministratorsOnly: true,
})
require.NoError(t, err)
return nil
})
require.NoError(t, err)
// User 2 created the template but an admin later changed it to admins-only
r := httptest.NewRequest(http.MethodDelete, "/custom_templates/1", nil)
r = mux.SetURLVars(r, map[string]string{"id": "1"})
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 2}))
rr := httptest.NewRecorder()
herr := handler.customTemplateDelete(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusForbidden, herr.StatusCode)
}
func TestCustomTemplateDelete_CreatorDeniedWithoutResourceControl(t *testing.T) {
t.Parallel()
handler, store, _ := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
CreatedByUserID: 2,
})
})
require.NoError(t, err)
// User 2 created this template but there is no resource control
r := httptest.NewRequest(http.MethodDelete, "/custom_templates/1", nil)
r = mux.SetURLVars(r, map[string]string{"id": "1"})
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 2}))
rr := httptest.NewRecorder()
herr := handler.customTemplateDelete(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusForbidden, herr.StatusCode)
}
func TestCustomTemplateDelete_Success(t *testing.T) {
t.Parallel()
handler, store, _ := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
err := tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
CreatedByUserID: 2,
})
require.NoError(t, err)
err = tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 1,
ResourceID: "1",
Type: portainer.CustomTemplateResourceControl,
UserAccesses: []portainer.UserResourceAccess{{UserID: 2}},
})
require.NoError(t, err)
return nil
})
require.NoError(t, err)
r := httptest.NewRequest(http.MethodDelete, "/custom_templates/1", nil)
r = mux.SetURLVars(r, map[string]string{"id": "1"})
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 2}))
rr := httptest.NewRecorder()
herr := handler.customTemplateDelete(rr, r)
require.Nil(t, herr)
require.Equal(t, http.StatusNoContent, rr.Code)
err = store.ViewTx(func(tx dataservices.DataStoreTx) error {
_, err := tx.CustomTemplate().Read(1)
require.True(t, tx.IsErrObjectNotFound(err))
return nil
})
require.NoError(t, err)
}
func TestCustomTemplateDelete_AdminCanDeleteAdminOnly(t *testing.T) {
t.Parallel()
handler, store, _ := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
err := tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
CreatedByUserID: 2,
})
require.NoError(t, err)
err = tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 1,
ResourceID: "1",
Type: portainer.CustomTemplateResourceControl,
AdministratorsOnly: true,
})
require.NoError(t, err)
return nil
})
require.NoError(t, err)
r := httptest.NewRequest(http.MethodDelete, "/custom_templates/1", nil)
r = mux.SetURLVars(r, map[string]string{"id": "1"})
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true}))
rr := httptest.NewRecorder()
herr := handler.customTemplateDelete(rr, r)
require.Nil(t, herr)
require.Equal(t, http.StatusNoContent, rr.Code)
}
func TestCustomTemplateDelete_PublicTemplateAllowsAnyUser(t *testing.T) {
t.Parallel()
handler, store, _ := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
err := tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
CreatedByUserID: 1,
})
require.NoError(t, err)
err = tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 1,
ResourceID: "1",
Type: portainer.CustomTemplateResourceControl,
Public: true,
})
require.NoError(t, err)
return nil
})
require.NoError(t, err)
// User 2 is not the creator but the template is public
r := httptest.NewRequest(http.MethodDelete, "/custom_templates/1", nil)
r = mux.SetURLVars(r, map[string]string{"id": "1"})
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 2}))
rr := httptest.NewRecorder()
herr := handler.customTemplateDelete(rr, r)
require.Nil(t, herr)
require.Equal(t, http.StatusNoContent, rr.Code)
}
func TestCustomTemplateDelete_NonCreatorForbiddenWithPrivateRC(t *testing.T) {
t.Parallel()
handler, store, _ := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
err := tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
CreatedByUserID: 1,
})
require.NoError(t, err)
err = tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 1,
ResourceID: "1",
Type: portainer.CustomTemplateResourceControl,
UserAccesses: []portainer.UserResourceAccess{{UserID: 1}},
})
require.NoError(t, err)
return nil
})
require.NoError(t, err)
// User 2 is not the creator and the template has a private resource control
r := httptest.NewRequest(http.MethodDelete, "/custom_templates/1", nil)
r = mux.SetURLVars(r, map[string]string{"id": "1"})
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 2}))
rr := httptest.NewRecorder()
herr := handler.customTemplateDelete(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusForbidden, herr.StatusCode)
}
func TestCustomTemplateDelete_CreatorDeniedWithoutAccess(t *testing.T) {
t.Parallel()
handler, store, _ := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
err := tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
CreatedByUserID: 2,
})
require.NoError(t, err)
// RC exists but only grants access to user 3, not the creator (user 2)
err = tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 1,
ResourceID: "1",
Type: portainer.CustomTemplateResourceControl,
UserAccesses: []portainer.UserResourceAccess{{UserID: 3}},
})
require.NoError(t, err)
return nil
})
require.NoError(t, err)
r := httptest.NewRequest(http.MethodDelete, "/custom_templates/1", nil)
r = mux.SetURLVars(r, map[string]string{"id": "1"})
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 2}))
rr := httptest.NewRecorder()
herr := handler.customTemplateDelete(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusForbidden, herr.StatusCode)
}

View File

@@ -59,12 +59,11 @@ func (handler *Handler) customTemplateFile(w http.ResponseWriter, r *http.Reques
return httperror.InternalServerError("Unable to retrieve user info from request context", err)
}
customTemplate.ResourceControl = resourceControl
canEdit := userCanEditTemplate(customTemplate, securityContext)
hasAccess := false
if resourceControl != nil {
customTemplate.ResourceControl = resourceControl
teamIDs := slicesx.Map(securityContext.UserMemberships, func(m portainer.TeamMembership) portainer.TeamID {
return m.TeamID
})
@@ -82,8 +81,8 @@ func (handler *Handler) customTemplateFile(w http.ResponseWriter, r *http.Reques
}
entryPath := customTemplate.EntryPoint
if customTemplate.GitConfig != nil {
entryPath = customTemplate.GitConfig.ConfigFilePath
if customTemplate.Artifact != nil && len(customTemplate.Artifact.Files) > 0 {
entryPath = customTemplate.Artifact.Files[0].Path
}
fileContent, err := handler.FileService.GetFileContent(customTemplate.ProjectPath, entryPath)
if err != nil {

View File

@@ -5,11 +5,13 @@ import (
"net/http/httptest"
"testing"
"github.com/gorilla/mux"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/http/security"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/gorilla/mux"
"github.com/segmentio/encoding/json"
"github.com/stretchr/testify/require"
)
@@ -33,7 +35,8 @@ func TestCustomTemplateFile(t *testing.T) {
require.NoError(t, err)
require.NoError(t, tx.CustomTemplate().Create(&portainer.CustomTemplate{ID: 2, EntryPoint: templateEntrypoint, ProjectPath: path}))
require.NoError(t, tx.ResourceControl().Create(&portainer.ResourceControl{ID: 1, ResourceID: "2", Type: portainer.CustomTemplateResourceControl,
require.NoError(t, tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 1, ResourceID: "2", Type: portainer.CustomTemplateResourceControl,
UserAccesses: []portainer.UserResourceAccess{{UserID: 2}},
TeamAccesses: []portainer.TeamResourceAccess{{TeamID: 1}},
}))
@@ -59,6 +62,7 @@ func TestCustomTemplateFile(t *testing.T) {
rr, r := test("1", &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
require.Nil(t, r)
require.Equal(t, http.StatusOK, rr.Result().StatusCode)
var res struct{ FileContent string }
require.NoError(t, json.NewDecoder(rr.Body).Decode(&res))
require.Equal(t, templateContent, res.FileContent)
@@ -74,6 +78,7 @@ func TestCustomTemplateFile(t *testing.T) {
rr, r := test("2", &security.RestrictedRequestContext{UserID: 2})
require.Nil(t, r)
require.Equal(t, http.StatusOK, rr.Result().StatusCode)
var res struct{ FileContent string }
require.NoError(t, json.NewDecoder(rr.Body).Decode(&res))
require.Equal(t, templateContent, res.FileContent)
@@ -83,6 +88,7 @@ func TestCustomTemplateFile(t *testing.T) {
rr, r := test("2", &security.RestrictedRequestContext{UserID: 3, UserMemberships: []portainer.TeamMembership{{ID: 1, UserID: 3, TeamID: 1}}})
require.Nil(t, r)
require.Equal(t, http.StatusOK, rr.Result().StatusCode)
var res struct{ FileContent string }
require.NoError(t, json.NewDecoder(rr.Body).Decode(&res))
require.Equal(t, templateContent, res.FileContent)
@@ -94,3 +100,85 @@ func TestCustomTemplateFile(t *testing.T) {
require.Equal(t, http.StatusForbidden, r.StatusCode)
})
}
func TestCustomTemplateFile_CreatorDeniedWhenAdminOnly(t *testing.T) {
t.Parallel()
handler, store, fs := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
path, err := fs.StoreCustomTemplateFileFromBytes("5", "entrypoint", []byte("content"))
require.NoError(t, err)
err = tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 5,
EntryPoint: "entrypoint",
ProjectPath: path,
CreatedByUserID: 2,
})
require.NoError(t, err)
err = tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 5,
ResourceID: "5",
Type: portainer.CustomTemplateResourceControl,
AdministratorsOnly: true,
})
require.NoError(t, err)
return nil
})
require.NoError(t, err)
r := httptest.NewRequest(http.MethodGet, "/custom_templates/5/file", nil)
r = mux.SetURLVars(r, map[string]string{"id": "5"})
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 2}))
rr := httptest.NewRecorder()
herr := handler.customTemplateFile(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusForbidden, herr.StatusCode)
}
func TestCustomTemplateFile_GitTemplate(t *testing.T) {
t.Parallel()
handler, ds, fs := newTestHandler(t)
templateContent := "git template content"
configFilePath := "docker-compose.yml"
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
src := &portainer.Source{
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{URL: "https://github.com/example/repo"},
}
err := tx.Source().Create(src)
require.NoError(t, err)
path, err := fs.StoreCustomTemplateFileFromBytes("10", configFilePath, []byte(templateContent))
require.NoError(t, err)
return tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 10,
EntryPoint: "should-not-be-used.yml",
ProjectPath: path,
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{{Path: configFilePath, SourceID: src.ID}},
},
})
}))
r := httptest.NewRequest(http.MethodGet, "/custom_templates/10/file", nil)
r = mux.SetURLVars(r, map[string]string{"id": "10"})
ctx := security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
r = r.WithContext(ctx)
rr := httptest.NewRecorder()
herr := handler.customTemplateFile(rr, r)
require.Nil(t, herr)
require.Equal(t, http.StatusOK, rr.Result().StatusCode)
var res struct{ FileContent string }
require.NoError(t, json.NewDecoder(rr.Body).Decode(&res))
require.Equal(t, templateContent, res.FileContent)
}

View File

@@ -7,6 +7,7 @@ import (
"sync"
portainer "github.com/portainer/portainer/api"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/stacks/stackutils"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
@@ -42,8 +43,28 @@ func (handler *Handler) customTemplateGitFetch(w http.ResponseWriter, r *http.Re
return httperror.InternalServerError("Unable to find a custom template with the specified identifier inside the database", err)
}
if customTemplate.GitConfig == nil {
return httperror.BadRequest("Git configuration does not exist in this custom template", err)
if customTemplate.Artifact == nil || len(customTemplate.Artifact.Files) == 0 {
return httperror.BadRequest("Git configuration does not exist in this custom template", nil)
}
file := customTemplate.Artifact.Files[0]
src, err := handler.DataStore.Source().Read(file.SourceID)
if err != nil {
return httperror.InternalServerError("Unable to retrieve git source for custom template", err)
}
if src.Git == nil {
return httperror.InternalServerError("Source has no git configuration", nil)
}
gitConfig := &gittypes.RepoConfig{
URL: src.Git.URL,
Authentication: src.Git.Authentication,
TLSSkipVerify: src.Git.TLSSkipVerify,
ReferenceName: file.Ref,
ConfigFilePath: file.Path,
ConfigHash: file.Hash,
}
// If multiple users are trying to fetch the same custom template simultaneously, a lock needs to be added
@@ -68,7 +89,7 @@ func (handler *Handler) customTemplateGitFetch(w http.ResponseWriter, r *http.Re
}
}()
commitHash, err := stackutils.DownloadGitRepository(context.TODO(), *customTemplate.GitConfig, handler.GitService, func() string {
commitHash, err := stackutils.DownloadGitRepository(context.TODO(), *gitConfig, handler.GitService, func() string {
return customTemplate.ProjectPath
})
if err != nil {
@@ -81,15 +102,15 @@ func (handler *Handler) customTemplateGitFetch(w http.ResponseWriter, r *http.Re
return httperror.InternalServerError("Failed to download git repository", err)
}
if customTemplate.GitConfig.ConfigHash != commitHash {
customTemplate.GitConfig.ConfigHash = commitHash
if customTemplate.Artifact.Files[0].Hash != commitHash {
customTemplate.Artifact.Files[0].Hash = commitHash
if err := handler.DataStore.CustomTemplate().Update(customTemplate.ID, customTemplate); err != nil {
return httperror.InternalServerError("Unable to persist custom template changes inside the database", err)
}
}
fileContent, err := handler.FileService.GetFileContent(customTemplate.ProjectPath, customTemplate.GitConfig.ConfigFilePath)
fileContent, err := handler.FileService.GetFileContent(customTemplate.ProjectPath, gitConfig.ConfigFilePath)
if err != nil {
return httperror.InternalServerError("Unable to retrieve custom template file from disk", err)
}

View File

@@ -156,6 +156,7 @@ func singleAPIRequest(h *Handler, jwt string, expect string) error {
func Test_customTemplateGitFetch(t *testing.T) {
t.Parallel()
is := assert.New(t)
_, store := datastore.MustNewTestStore(t, true, true)
@@ -172,12 +173,31 @@ func Test_customTemplateGitFetch(t *testing.T) {
dir, err := os.Getwd()
require.NoError(t, err, "error to get working directory")
template1 := &portainer.CustomTemplate{ID: 1, Title: "custom-template-1", ProjectPath: filesystem.JoinPaths(dir, "fixtures/custom_template_1"), GitConfig: &gittypes.RepoConfig{ConfigFilePath: "test-config-path.txt"}}
src := &portainer.Source{
ID: 1,
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: "https://github.com/example/repo",
},
}
err = store.Source().Create(src)
require.NoError(t, err, "error creating source")
const configFilePath = "test-config-path.txt"
template1 := &portainer.CustomTemplate{
ID: 1,
Title: "custom-template-1",
ProjectPath: filesystem.JoinPaths(dir, "fixtures/custom_template_1"),
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{{Path: configFilePath, SourceID: src.ID}},
},
}
err = store.CustomTemplateService.Create(template1)
require.NoError(t, err, "error creating custom template 1")
// prepare testing folder
err = prepareTestFolder(template1.ProjectPath, template1.GitConfig.ConfigFilePath)
err = prepareTestFolder(template1.ProjectPath, configFilePath)
require.NoError(t, err, "error creating testing folder")
defer func() {
@@ -192,7 +212,7 @@ func Test_customTemplateGitFetch(t *testing.T) {
requestBouncer := security.NewRequestBouncer(t.Context(), store, jwtService, nil)
gitService := &TestGitService{
targetFilePath: filesystem.JoinPaths(template1.ProjectPath, template1.GitConfig.ConfigFilePath),
targetFilePath: filesystem.JoinPaths(template1.ProjectPath, configFilePath),
}
fileService := &TestFileService{}
@@ -252,7 +272,7 @@ func Test_customTemplateGitFetch(t *testing.T) {
t.Run("restore git repository if it is failed to download the new git repository", func(t *testing.T) {
invalidGitService := &InvalidTestGitService{
targetFilePath: filesystem.JoinPaths(template1.ProjectPath, template1.GitConfig.ConfigFilePath),
targetFilePath: filesystem.JoinPaths(template1.ProjectPath, configFilePath),
}
h := NewHandler(requestBouncer, store, fileService, invalidGitService)
@@ -274,3 +294,73 @@ func Test_customTemplateGitFetch(t *testing.T) {
assert.Equal(t, "gfedcba", string(fileContent))
})
}
func TestCustomTemplateGitFetch_NilArtifactReturnsBadRequest(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
template := &portainer.CustomTemplate{ID: 1, Title: "no-git-template"}
err := store.CustomTemplateService.Create(template)
require.NoError(t, err)
h := NewHandler(testhelpers.NewTestRequestBouncer(), store, &TestFileService{}, &TestGitService{})
req := httptest.NewRequest(http.MethodPut, "/custom_templates/1/git_fetch", bytes.NewBufferString("{}"))
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
require.Equal(t, http.StatusBadRequest, rr.Code)
}
func TestCustomTemplateGitFetch_EmptySourceIDsReturnsBadRequest(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
template := &portainer.CustomTemplate{
ID: 1,
Title: "empty-source-ids",
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{},
},
}
err := store.CustomTemplateService.Create(template)
require.NoError(t, err)
h := NewHandler(testhelpers.NewTestRequestBouncer(), store, &TestFileService{}, &TestGitService{})
req := httptest.NewRequest(http.MethodPut, "/custom_templates/1/git_fetch", bytes.NewBufferString("{}"))
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
require.Equal(t, http.StatusBadRequest, rr.Code)
}
func TestCustomTemplateGitFetch_SourceWithNilGitConfigReturnsInternalError(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
src := &portainer.Source{Type: portainer.SourceTypeGit}
err := store.Source().Create(src)
require.NoError(t, err)
template := &portainer.CustomTemplate{
ID: 1,
Title: "nil-git-config",
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{{SourceID: src.ID}},
},
}
err = store.CustomTemplateService.Create(template)
require.NoError(t, err)
h := NewHandler(testhelpers.NewTestRequestBouncer(), store, &TestFileService{}, &TestGitService{})
req := httptest.NewRequest(http.MethodPut, "/custom_templates/1/git_fetch", bytes.NewBufferString("{}"))
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
require.Equal(t, http.StatusInternalServerError, rr.Code)
}

View File

@@ -54,25 +54,25 @@ func (handler *Handler) customTemplateInspect(w http.ResponseWriter, r *http.Req
return httperror.InternalServerError("Unable to retrieve user info from request context", err)
}
customTemplate.ResourceControl = resourceControl
canEdit := userCanEditTemplate(customTemplate, securityContext)
hasAccess := false
if resourceControl != nil {
customTemplate.ResourceControl = resourceControl
teamIDs := slicesx.Map(securityContext.UserMemberships, func(m portainer.TeamMembership) portainer.TeamID {
return m.TeamID
})
hasAccess = authorization.UserCanAccessResource(securityContext.UserID, teamIDs, resourceControl)
}
if canEdit || hasAccess {
return nil
if !canEdit && !hasAccess {
return httperror.Forbidden("Access denied to resource", httperrors.ErrResourceAccessDenied)
}
return httperror.Forbidden("Access denied to resource", httperrors.ErrResourceAccessDenied)
populateGitConfig(tx, customTemplate)
return nil
})
return response.TxResponse(w, customTemplate, err)

View File

@@ -5,14 +5,16 @@ import (
"net/http/httptest"
"testing"
"github.com/gorilla/mux"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/datastore"
"github.com/portainer/portainer/api/filesystem"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/http/security"
"github.com/portainer/portainer/api/internal/testhelpers"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/gorilla/mux"
"github.com/segmentio/encoding/json"
"github.com/stretchr/testify/require"
)
@@ -33,11 +35,13 @@ func newTestHandler(t *testing.T) (*Handler, dataservices.DataStore, portainer.F
require.NoError(t, tx.User().Create(&portainer.User{ID: 2, Username: "std2", Role: portainer.StandardUserRole}))
require.NoError(t, tx.User().Create(&portainer.User{ID: 3, Username: "std3", Role: portainer.StandardUserRole}))
require.NoError(t, tx.User().Create(&portainer.User{ID: 4, Username: "std4", Role: portainer.StandardUserRole}))
require.NoError(t, tx.Endpoint().Create(&portainer.Endpoint{ID: 1,
require.NoError(t, tx.Endpoint().Create(&portainer.Endpoint{
ID: 1,
UserAccessPolicies: portainer.UserAccessPolicies{
2: portainer.AccessPolicy{RoleID: 0},
3: portainer.AccessPolicy{RoleID: 0},
}}))
},
}))
require.NoError(t, tx.Team().Create(&portainer.Team{ID: 1}))
require.NoError(t, tx.TeamMembership().Create(&portainer.TeamMembership{ID: 1, UserID: 3, TeamID: 1, Role: portainer.TeamMember}))
return nil
@@ -56,7 +60,8 @@ func TestInspectHandler(t *testing.T) {
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
require.NoError(t, tx.CustomTemplate().Create(&portainer.CustomTemplate{ID: 1}))
require.NoError(t, tx.CustomTemplate().Create(&portainer.CustomTemplate{ID: 2}))
require.NoError(t, tx.ResourceControl().Create(&portainer.ResourceControl{ID: 1, ResourceID: "2", Type: portainer.CustomTemplateResourceControl,
require.NoError(t, tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 1, ResourceID: "2", Type: portainer.CustomTemplateResourceControl,
UserAccesses: []portainer.UserResourceAccess{{UserID: 2}},
TeamAccesses: []portainer.TeamResourceAccess{{TeamID: 1}},
}))
@@ -82,6 +87,7 @@ func TestInspectHandler(t *testing.T) {
rr, r := test("1", &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
require.Nil(t, r)
require.Equal(t, http.StatusOK, rr.Result().StatusCode)
var template portainer.CustomTemplate
require.NoError(t, json.NewDecoder(rr.Body).Decode(&template))
require.Equal(t, portainer.CustomTemplateID(1), template.ID)
@@ -97,6 +103,7 @@ func TestInspectHandler(t *testing.T) {
rr, r := test("2", &security.RestrictedRequestContext{UserID: 2})
require.Nil(t, r)
require.Equal(t, http.StatusOK, rr.Result().StatusCode)
var template portainer.CustomTemplate
require.NoError(t, json.NewDecoder(rr.Body).Decode(&template))
require.Equal(t, portainer.CustomTemplateID(2), template.ID)
@@ -106,6 +113,7 @@ func TestInspectHandler(t *testing.T) {
rr, r := test("2", &security.RestrictedRequestContext{UserID: 3, UserMemberships: []portainer.TeamMembership{{ID: 1, UserID: 3, TeamID: 1}}})
require.Nil(t, r)
require.Equal(t, http.StatusOK, rr.Result().StatusCode)
var template portainer.CustomTemplate
require.NoError(t, json.NewDecoder(rr.Body).Decode(&template))
require.Equal(t, portainer.CustomTemplateID(2), template.ID)
@@ -117,3 +125,88 @@ func TestInspectHandler(t *testing.T) {
require.Equal(t, http.StatusForbidden, r.StatusCode)
})
}
func TestInspectHandler_CreatorDeniedWhenAdminOnly(t *testing.T) {
t.Parallel()
handler, store, _ := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
err := tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 5,
CreatedByUserID: 2,
})
require.NoError(t, err)
err = tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 5,
ResourceID: "5",
Type: portainer.CustomTemplateResourceControl,
AdministratorsOnly: true,
})
require.NoError(t, err)
return nil
})
require.NoError(t, err)
r := httptest.NewRequest(http.MethodGet, "/custom_templates/5", nil)
r = mux.SetURLVars(r, map[string]string{"id": "5"})
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 2}))
rr := httptest.NewRecorder()
herr := handler.customTemplateInspect(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusForbidden, herr.StatusCode)
}
func TestInspectHandler_GitConfigPopulatedFromSource(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
var srcID portainer.SourceID
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
src := &portainer.Source{
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: "https://github.com/example/repo",
TLSSkipVerify: true,
},
}
err := tx.Source().Create(src)
require.NoError(t, err)
srcID = src.ID
return tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 10,
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{{
Ref: "refs/heads/main",
Path: "docker-compose.yml",
Hash: "abc123",
SourceID: srcID,
}},
},
})
}))
r := httptest.NewRequest(http.MethodGet, "/custom_templates/10", nil)
r = mux.SetURLVars(r, map[string]string{"id": "10"})
ctx := security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
r = r.WithContext(ctx)
rr := httptest.NewRecorder()
herr := handler.customTemplateInspect(rr, r)
require.Nil(t, herr)
require.Equal(t, http.StatusOK, rr.Result().StatusCode)
var template portainer.CustomTemplate
require.NoError(t, json.NewDecoder(rr.Body).Decode(&template))
require.NotNil(t, template.GitConfig)
require.Equal(t, "https://github.com/example/repo", template.GitConfig.URL)
require.True(t, template.GitConfig.TLSSkipVerify)
require.Equal(t, "refs/heads/main", template.GitConfig.ReferenceName)
require.Equal(t, "docker-compose.yml", template.GitConfig.ConfigFilePath)
require.Equal(t, "abc123", template.GitConfig.ConfigHash)
}

View File

@@ -74,10 +74,7 @@ func (handler *Handler) customTemplateList(w http.ResponseWriter, r *http.Reques
}
for i := range customTemplates {
customTemplate := &customTemplates[i]
if customTemplate.GitConfig != nil && customTemplate.GitConfig.Authentication != nil {
customTemplate.GitConfig.Authentication.Password = ""
}
populateGitConfig(handler.DataStore, &customTemplates[i])
}
return response.JSON(w, customTemplates)

View File

@@ -0,0 +1,127 @@
package customtemplates
import (
"net/http"
"net/http/httptest"
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/http/security"
"github.com/segmentio/encoding/json"
"github.com/stretchr/testify/require"
)
func TestCustomTemplateList_PopulatesGitConfigFromSource(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
var srcID portainer.SourceID
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
src := &portainer.Source{
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: "https://github.com/example/repo",
TLSSkipVerify: true,
},
}
err := tx.Source().Create(src)
require.NoError(t, err)
srcID = src.ID
require.NoError(t, tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{{
Ref: "refs/heads/main",
Path: "docker-compose.yml",
Hash: "abc123",
SourceID: srcID,
}},
},
}))
require.NoError(t, tx.CustomTemplate().Create(&portainer.CustomTemplate{ID: 2, EntryPoint: "docker-compose.yml"}))
return nil
}))
r := httptest.NewRequest(http.MethodGet, "/custom_templates", nil)
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true}))
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, r)
require.Equal(t, http.StatusOK, rr.Code)
var templates []portainer.CustomTemplate
require.NoError(t, json.NewDecoder(rr.Body).Decode(&templates))
var gitTemplate portainer.CustomTemplate
for _, tpl := range templates {
if tpl.ID == 1 {
gitTemplate = tpl
}
}
require.NotNil(t, gitTemplate.GitConfig)
require.Equal(t, "https://github.com/example/repo", gitTemplate.GitConfig.URL)
require.True(t, gitTemplate.GitConfig.TLSSkipVerify)
require.Equal(t, "refs/heads/main", gitTemplate.GitConfig.ReferenceName)
require.Equal(t, "docker-compose.yml", gitTemplate.GitConfig.ConfigFilePath)
require.Equal(t, "abc123", gitTemplate.GitConfig.ConfigHash)
var plainTemplate portainer.CustomTemplate
for _, tpl := range templates {
if tpl.ID == 2 {
plainTemplate = tpl
}
}
require.Nil(t, plainTemplate.GitConfig)
}
func TestCustomTemplateList_StripsPasswordFromGitConfig(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
var srcID portainer.SourceID
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
src := &portainer.Source{
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: "https://github.com/example/repo",
Authentication: &gittypes.GitAuthentication{
Username: "user",
Password: "topsecret",
},
},
}
err := tx.Source().Create(src)
require.NoError(t, err)
srcID = src.ID
require.NoError(t, tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{{SourceID: srcID}},
},
}))
return nil
}))
r := httptest.NewRequest(http.MethodGet, "/custom_templates", nil)
r = r.WithContext(security.StoreRestrictedRequestContext(r, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true}))
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, r)
require.Equal(t, http.StatusOK, rr.Code)
var templates []portainer.CustomTemplate
require.NoError(t, json.NewDecoder(rr.Body).Decode(&templates))
require.Len(t, templates, 1)
require.NotNil(t, templates[0].GitConfig)
require.NotNil(t, templates[0].GitConfig.Authentication)
require.Equal(t, "user", templates[0].GitConfig.Authentication.Username)
require.Empty(t, templates[0].GitConfig.Authentication.Password)
}

View File

@@ -8,9 +8,12 @@ import (
"strconv"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/filesystem"
"github.com/portainer/portainer/api/git"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/gitops/sources"
"github.com/portainer/portainer/api/gitops/workflows"
httperrors "github.com/portainer/portainer/api/http/errors"
"github.com/portainer/portainer/api/http/security"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
@@ -34,28 +37,26 @@ type customTemplateUpdatePayload struct {
Platform portainer.CustomTemplatePlatform `example:"1" enums:"1,2"`
// Type of created stack (1 - swarm, 2 - compose, 3 - kubernetes)
Type portainer.StackType `example:"1" enums:"1,2,3" validate:"required"`
// URL of a Git repository hosting the Stack file
RepositoryURL string `example:"https://github.com/openfaas/faas" validate:"required"`
// SourceID references an existing Source for git credentials/URL.
// When set, the inline URL and authentication fields are ignored.
SourceID portainer.SourceID `example:"1"`
// Deprecated: use SourceID instead. URL of a Git repository hosting the Stack file.
RepositoryURL string `example:"https://github.com/openfaas/faas"`
// Reference name of a Git repository hosting the Stack file
RepositoryReferenceName string `example:"refs/heads/master"`
// Use authentication to clone the Git repository
// Deprecated: use SourceID instead. Use authentication to clone the Git repository.
RepositoryAuthentication bool `example:"true"`
// Username used in basic authentication. Required when RepositoryAuthentication is true
// and RepositoryGitCredentialID is 0. Ignored if RepositoryAuthType is token
// Deprecated: use SourceID instead. Username used in basic authentication. Required when RepositoryAuthentication is true.
RepositoryUsername string `example:"myGitUsername"`
// Password used in basic authentication or token used in token authentication.
// Required when RepositoryAuthentication is true and RepositoryGitCredentialID is 0
// Deprecated: use SourceID instead. Password used in basic authentication or token used in token authentication. Required when RepositoryAuthentication is true.
RepositoryPassword string `example:"myGitPassword"`
// GitCredentialID used to identify the bound git credential. Required when RepositoryAuthentication
// is true and RepositoryUsername/RepositoryPassword are not provided
RepositoryGitCredentialID int `example:"0"`
// Path to the Stack file inside the Git repository
ComposeFilePathInRepository string `example:"docker-compose.yml" default:"docker-compose.yml"`
// Content of stack file
FileContent string `validate:"required"`
// Definitions of variables in the stack file
Variables []portainer.CustomTemplateVariableDefinition
// TLSSkipVerify skips SSL verification when cloning the Git repository
// Deprecated: use SourceID instead. TLSSkipVerify skips SSL verification when cloning the Git repository.
TLSSkipVerify bool `example:"false"`
// IsComposeFormat indicates if the Kubernetes template is created from a Docker Compose file
IsComposeFormat bool `example:"false"`
@@ -68,10 +69,6 @@ func (payload *customTemplateUpdatePayload) Validate(r *http.Request) error {
return errors.New("Invalid custom template title")
}
if len(payload.FileContent) == 0 && len(payload.RepositoryURL) == 0 {
return errors.New("Either file content or git repository url need to be provided")
}
if payload.Type != portainer.KubernetesStack && payload.Platform != portainer.CustomTemplatePlatformLinux && payload.Platform != portainer.CustomTemplatePlatformWindows {
return errors.New("Invalid custom template platform")
}
@@ -84,19 +81,30 @@ func (payload *customTemplateUpdatePayload) Validate(r *http.Request) error {
return errors.New("Invalid custom template description")
}
if !isValidNote(payload.Note) {
if !IsValidNote(payload.Note) {
return errors.New("Invalid note. <img> tag is not supported")
}
if payload.RepositoryAuthentication && (len(payload.RepositoryUsername) == 0 || len(payload.RepositoryPassword) == 0) {
return errors.New("Invalid repository credentials. Username and password must be specified when authentication is enabled")
if len(payload.FileContent) == 0 && payload.SourceID == 0 {
if len(payload.RepositoryURL) == 0 {
return errors.New("Either file content, git repository url, or source ID need to be provided")
}
if !validate.IsURL(payload.RepositoryURL) {
return errors.New("Invalid repository URL. Must correspond to a valid URL format")
}
if payload.RepositoryAuthentication && (len(payload.RepositoryUsername) == 0 || len(payload.RepositoryPassword) == 0) {
return errors.New("Invalid repository credentials. Username and password must be specified when authentication is enabled")
}
}
if len(payload.ComposeFilePathInRepository) == 0 {
payload.ComposeFilePathInRepository = filesystem.ComposeFileDefaultName
}
if err := validateVariablesDefinitions(payload.Variables); err != nil {
if err := ValidateVariablesDefinitions(payload.Variables); err != nil {
return err
}
@@ -131,15 +139,15 @@ func (handler *Handler) customTemplateUpdate(w http.ResponseWriter, r *http.Requ
return httperror.BadRequest("Invalid request payload", err)
}
customTemplates, err := handler.DataStore.CustomTemplate().ReadAll()
duplicates, err := handler.DataStore.CustomTemplate().ReadAll(func(t portainer.CustomTemplate) bool {
return t.ID != portainer.CustomTemplateID(customTemplateID) && t.Title == payload.Title
})
if err != nil {
return httperror.InternalServerError("Unable to retrieve custom templates from the database", err)
}
for _, existingTemplate := range customTemplates {
if existingTemplate.ID != portainer.CustomTemplateID(customTemplateID) && existingTemplate.Title == payload.Title {
return httperror.InternalServerError("Template name must be unique", errors.New("Template name must be unique"))
}
if len(duplicates) > 0 {
return httperror.InternalServerError("Template name must be unique", errors.New("Template name must be unique"))
}
customTemplate, err := handler.DataStore.CustomTemplate().Read(portainer.CustomTemplateID(customTemplateID))
@@ -154,8 +162,13 @@ func (handler *Handler) customTemplateUpdate(w http.ResponseWriter, r *http.Requ
return httperror.InternalServerError("Unable to retrieve info from request context", err)
}
access := userCanEditTemplate(customTemplate, securityContext)
if !access {
resourceControl, err := handler.DataStore.ResourceControl().ResourceControlByResourceIDAndType(strconv.Itoa(customTemplateID), portainer.CustomTemplateResourceControl)
if err != nil {
return httperror.InternalServerError("Unable to retrieve a resource control associated to the custom template", err)
}
customTemplate.ResourceControl = resourceControl
if !userCanEditTemplate(customTemplate, securityContext) {
return httperror.Forbidden("Access denied to resource", httperrors.ErrResourceAccessDenied)
}
@@ -169,35 +182,33 @@ func (handler *Handler) customTemplateUpdate(w http.ResponseWriter, r *http.Requ
customTemplate.IsComposeFormat = payload.IsComposeFormat
customTemplate.EdgeTemplate = payload.EdgeTemplate
if payload.RepositoryURL != "" {
if !validate.IsURL(payload.RepositoryURL) {
return httperror.BadRequest("Invalid repository URL. Must correspond to a valid URL format", err)
if payload.SourceID != 0 || payload.RepositoryURL != "" {
gitConfig, httpErr := sources.ResolveRepoConfig(handler.DataStore, sources.RepoConfigInput{
SourceID: payload.SourceID,
ReferenceName: payload.RepositoryReferenceName,
ConfigFilePath: payload.ComposeFilePathInRepository,
RepositoryURL: payload.RepositoryURL,
TLSSkipVerify: payload.TLSSkipVerify,
RepositoryAuthentication: payload.RepositoryAuthentication,
Username: payload.RepositoryUsername,
Password: payload.RepositoryPassword,
})
if httpErr != nil {
return httpErr
}
gitConfig := &gittypes.RepoConfig{
URL: payload.RepositoryURL,
ReferenceName: payload.RepositoryReferenceName,
ConfigFilePath: payload.ComposeFilePathInRepository,
TLSSkipVerify: payload.TLSSkipVerify,
}
repositoryUsername := ""
repositoryPassword := ""
if payload.RepositoryAuthentication {
repositoryUsername = payload.RepositoryUsername
repositoryPassword = payload.RepositoryPassword
gitConfig.Authentication = &gittypes.GitAuthentication{
Username: payload.RepositoryUsername,
Password: payload.RepositoryPassword,
}
var username, password string
if gitConfig.Authentication != nil {
username = gitConfig.Authentication.Username
password = gitConfig.Authentication.Password
}
cleanBackup, err := git.CloneWithBackup(context.TODO(), handler.GitService, handler.FileService, git.CloneOptions{
ProjectPath: customTemplate.ProjectPath,
URL: gitConfig.URL,
ReferenceName: gitConfig.ReferenceName,
Username: repositoryUsername,
Password: repositoryPassword,
Username: username,
Password: password,
TLSSkipVerify: gitConfig.TLSSkipVerify,
})
if err != nil {
@@ -210,16 +221,40 @@ func (handler *Handler) customTemplateUpdate(w http.ResponseWriter, r *http.Requ
context.TODO(),
gitConfig.URL,
gitConfig.ReferenceName,
repositoryUsername,
repositoryPassword,
username,
password,
gitConfig.TLSSkipVerify,
)
if err != nil {
return httperror.InternalServerError("Unable get latest commit id", fmt.Errorf("failed to fetch latest commit id of the template %v: %w", customTemplate.ID, err))
}
gitConfig.ConfigHash = commitHash
customTemplate.GitConfig = gitConfig
sourceID := payload.SourceID
if sourceID == 0 {
src, err := workflows.FindOrCreateGitSource(handler.DataStore, &portainer.Source{
Name: gittypes.RepoName(gitConfig.URL),
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: gitConfig.URL,
Authentication: gitConfig.Authentication,
TLSSkipVerify: gitConfig.TLSSkipVerify,
},
})
if err != nil {
return httperror.InternalServerError("Unable to find or create git source", err)
}
sourceID = src.ID
}
customTemplate.Artifact = &portainer.Artifact{
Files: []portainer.ArtifactFile{{
SourceID: sourceID,
Path: gitConfig.ConfigFilePath,
Ref: gitConfig.ReferenceName,
Hash: commitHash,
}},
}
} else {
templateFolder := strconv.Itoa(customTemplateID)
projectPath, err := handler.FileService.StoreCustomTemplateFileFromBytes(templateFolder, customTemplate.EntryPoint, []byte(payload.FileContent))
@@ -228,11 +263,18 @@ func (handler *Handler) customTemplateUpdate(w http.ResponseWriter, r *http.Requ
}
customTemplate.ProjectPath = projectPath
customTemplate.Artifact = nil
}
if err := handler.DataStore.CustomTemplate().Update(customTemplate.ID, customTemplate); err != nil {
return httperror.InternalServerError("Unable to persist custom template changes inside the database", err)
}
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
if err := tx.CustomTemplate().Update(customTemplate.ID, customTemplate); err != nil {
return httperror.InternalServerError("Unable to persist custom template changes inside the database", err)
}
return response.JSON(w, customTemplate)
populateGitConfig(tx, customTemplate)
return nil
})
return response.TxResponse(w, customTemplate, err)
}

View File

@@ -0,0 +1,641 @@
package customtemplates
import (
"bytes"
"net/http"
"net/http/httptest"
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/filesystem"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/http/security"
"github.com/gorilla/mux"
"github.com/segmentio/encoding/json"
"github.com/stretchr/testify/require"
)
func updateTemplateRequest(t *testing.T, templateID string, payload any, ctx *security.RestrictedRequestContext) *http.Request {
t.Helper()
body, err := json.Marshal(payload)
require.NoError(t, err)
r := httptest.NewRequest(http.MethodPut, "/custom_templates/"+templateID, bytes.NewReader(body))
r.Header.Set("Content-Type", "application/json")
r = mux.SetURLVars(r, map[string]string{"id": templateID})
return r.WithContext(security.StoreRestrictedRequestContext(r, ctx))
}
func TestCustomTemplateUpdate_NotFound(t *testing.T) {
t.Parallel()
handler, _, _ := newTestHandler(t)
payload := customTemplateUpdatePayload{
Title: "New Title",
Description: "New Description",
FileContent: "version: '3'",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "99", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusNotFound, herr.StatusCode)
}
func TestCustomTemplateUpdate_Forbidden(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Title: "Original Title",
EntryPoint: filesystem.ComposeFileDefaultName,
CreatedByUserID: 1,
})
}))
payload := customTemplateUpdatePayload{
Title: "New Title",
Description: "New Description",
FileContent: "version: '3'",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
// User 2 did not create this template and is not an admin
r := updateTemplateRequest(t, "1", payload, &security.RestrictedRequestContext{UserID: 2})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusForbidden, herr.StatusCode)
}
func TestCustomTemplateUpdate_DuplicateTitle(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
require.NoError(t, tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Title: "Template One",
}))
return tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 2,
Title: "Template Two",
})
}))
payload := customTemplateUpdatePayload{
Title: "Template One",
Description: "Renamed",
FileContent: "version: '3'",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "2", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusInternalServerError, herr.StatusCode)
}
func TestCustomTemplateUpdate_Success_FileContent(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Title: "Original Title",
Description: "Original Description",
EntryPoint: filesystem.ComposeFileDefaultName,
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
CreatedByUserID: 1,
})
}))
payload := customTemplateUpdatePayload{
Title: "Updated Title",
Description: "Updated Description",
FileContent: "version: '3'\nservices:\n app:\n image: alpine",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "1", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.Nil(t, herr)
require.Equal(t, http.StatusOK, rr.Code)
var tmpl portainer.CustomTemplate
require.NoError(t, json.NewDecoder(rr.Body).Decode(&tmpl))
require.Equal(t, "Updated Title", tmpl.Title)
require.Equal(t, "Updated Description", tmpl.Description)
err := ds.ViewTx(func(tx dataservices.DataStoreTx) error {
stored, err := tx.CustomTemplate().Read(1)
require.NoError(t, err)
require.Equal(t, "Updated Title", stored.Title)
require.Equal(t, "Updated Description", stored.Description)
return nil
})
require.NoError(t, err)
}
func TestCustomTemplateUpdate_SameTitleAllowed(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Title: "My Template",
EntryPoint: filesystem.ComposeFileDefaultName,
})
}))
payload := customTemplateUpdatePayload{
Title: "My Template",
Description: "Updated description",
FileContent: "version: '3'",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "1", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.Nil(t, herr)
require.Equal(t, http.StatusOK, rr.Code)
err := ds.ViewTx(func(tx dataservices.DataStoreTx) error {
stored, err := tx.CustomTemplate().Read(1)
require.NoError(t, err)
require.Equal(t, "My Template", stored.Title)
require.Equal(t, "Updated description", stored.Description)
return nil
})
require.NoError(t, err)
}
func TestCustomTemplateUpdate_InvalidPayload(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Title: "My Template",
EntryPoint: filesystem.ComposeFileDefaultName,
})
}))
payload := customTemplateUpdatePayload{
// Title is empty - invalid
Description: "A description",
FileContent: "version: '3'",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "1", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusBadRequest, herr.StatusCode)
}
func TestCustomTemplateUpdate_Validation_MissingDescription(t *testing.T) {
t.Parallel()
handler, _, _ := newTestHandler(t)
payload := customTemplateUpdatePayload{
Title: "My Template",
FileContent: "version: '3'",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "99", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusBadRequest, herr.StatusCode)
}
func TestCustomTemplateUpdate_Validation_BothContentAndRepoMissing(t *testing.T) {
t.Parallel()
handler, _, _ := newTestHandler(t)
payload := customTemplateUpdatePayload{
Title: "My Template",
Description: "A description",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "99", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusBadRequest, herr.StatusCode)
}
func TestCustomTemplateUpdate_Validation_InvalidPlatform(t *testing.T) {
t.Parallel()
handler, _, _ := newTestHandler(t)
payload := customTemplateUpdatePayload{
Title: "My Template",
Description: "A description",
FileContent: "version: '3'",
Type: portainer.DockerComposeStack,
Platform: 0,
}
r := updateTemplateRequest(t, "99", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusBadRequest, herr.StatusCode)
}
func TestCustomTemplateUpdate_Validation_InvalidType(t *testing.T) {
t.Parallel()
handler, _, _ := newTestHandler(t)
payload := customTemplateUpdatePayload{
Title: "My Template",
Description: "A description",
FileContent: "version: '3'",
Type: 0,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "99", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusBadRequest, herr.StatusCode)
}
func TestCustomTemplateUpdate_Validation_NoteWithImage(t *testing.T) {
t.Parallel()
handler, _, _ := newTestHandler(t)
payload := customTemplateUpdatePayload{
Title: "My Template",
Description: "A description",
FileContent: "version: '3'",
Note: `Some note <img src="x" onerror="alert(1)">`,
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "99", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusBadRequest, herr.StatusCode)
}
func TestCustomTemplateUpdate_Validation_AuthWithoutCredentials(t *testing.T) {
t.Parallel()
handler, _, _ := newTestHandler(t)
payload := customTemplateUpdatePayload{
Title: "My Template",
Description: "A description",
RepositoryURL: "https://github.com/example/repo",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
RepositoryAuthentication: true,
}
r := updateTemplateRequest(t, "99", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusBadRequest, herr.StatusCode)
}
func TestCustomTemplateUpdate_ClearsArtifact(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Title: "Git Template",
EntryPoint: filesystem.ComposeFileDefaultName,
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
CreatedByUserID: 1,
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{},
},
})
}))
payload := customTemplateUpdatePayload{
Title: "Git Template",
Description: "Updated with file content",
FileContent: "version: '3'\nservices:\n app:\n image: alpine",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "1", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.Nil(t, herr)
require.Equal(t, http.StatusOK, rr.Code)
var tmpl portainer.CustomTemplate
require.NoError(t, json.NewDecoder(rr.Body).Decode(&tmpl))
require.Nil(t, tmpl.Artifact)
err := ds.ViewTx(func(tx dataservices.DataStoreTx) error {
stored, err := tx.CustomTemplate().Read(1)
require.NoError(t, err)
require.Nil(t, stored.Artifact)
return nil
})
require.NoError(t, err)
}
func TestCustomTemplateUpdate_CreatorDeniedWhenAdminOnly(t *testing.T) {
t.Parallel()
handler, store, _ := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
err := tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Title: "User Template",
EntryPoint: filesystem.ComposeFileDefaultName,
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
CreatedByUserID: 2,
})
require.NoError(t, err)
err = tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 1,
ResourceID: "1",
Type: portainer.CustomTemplateResourceControl,
AdministratorsOnly: true,
})
require.NoError(t, err)
return nil
})
require.NoError(t, err)
payload := customTemplateUpdatePayload{
Title: "User Template Updated",
Description: "Attempted update by creator after adminonly change",
FileContent: "version: '3'",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "1", payload, &security.RestrictedRequestContext{UserID: 2})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusForbidden, herr.StatusCode)
}
func TestCustomTemplateUpdate_WithSourceID_Success(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
handler.GitService = &gitServiceCreatingFile{}
projectDir := t.TempDir()
var srcID portainer.SourceID
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
require.NoError(t, tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Title: "Source Template",
EntryPoint: filesystem.ComposeFileDefaultName,
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
CreatedByUserID: 1,
ProjectPath: projectDir,
}))
src := &portainer.Source{
Name: "example/repo",
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: "https://github.com/example/repo",
},
}
err := tx.Source().Create(src)
require.NoError(t, err)
srcID = src.ID
return nil
}))
payload := customTemplateUpdatePayload{
Title: "Source Template",
Description: "Updated via source ID",
SourceID: srcID,
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "1", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.Nil(t, herr)
require.Equal(t, http.StatusOK, rr.Code)
var tmpl portainer.CustomTemplate
require.NoError(t, json.NewDecoder(rr.Body).Decode(&tmpl))
require.NotNil(t, tmpl.Artifact)
require.Len(t, tmpl.Artifact.Files, 1)
require.Equal(t, srcID, tmpl.Artifact.Files[0].SourceID)
require.Equal(t, "deadbeef123", tmpl.Artifact.Files[0].Hash)
}
func TestCustomTemplateUpdate_WithSourceID_NonExistentSource(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
handler.GitService = &gitServiceCreatingFile{}
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Title: "Source Template",
EntryPoint: filesystem.ComposeFileDefaultName,
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
CreatedByUserID: 1,
})
}))
payload := customTemplateUpdatePayload{
Title: "Source Template",
Description: "Updated via non-existent source ID",
SourceID: 999,
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "1", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.NotNil(t, herr)
require.Equal(t, http.StatusNotFound, herr.StatusCode)
}
func TestCustomTemplateUpdate_AdminCanUpdateAdminOnly(t *testing.T) {
t.Parallel()
handler, store, _ := newTestHandler(t)
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
err := tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Title: "User Template",
EntryPoint: filesystem.ComposeFileDefaultName,
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
CreatedByUserID: 2,
})
require.NoError(t, err)
err = tx.ResourceControl().Create(&portainer.ResourceControl{
ID: 1,
ResourceID: "1",
Type: portainer.CustomTemplateResourceControl,
AdministratorsOnly: true,
})
require.NoError(t, err)
return nil
})
require.NoError(t, err)
payload := customTemplateUpdatePayload{
Title: "Updated by Admin",
Description: "Admin update of adminonly template",
FileContent: "version: '3'",
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "1", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.Nil(t, herr)
require.Equal(t, http.StatusOK, rr.Code)
}
func TestCustomTemplateUpdate_GitRepository_Success(t *testing.T) {
t.Parallel()
handler, ds, _ := newTestHandler(t)
handler.GitService = &gitServiceCreatingFile{}
projectDir := t.TempDir()
require.NoError(t, ds.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.CustomTemplate().Create(&portainer.CustomTemplate{
ID: 1,
Title: "Git Template",
EntryPoint: filesystem.ComposeFileDefaultName,
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
CreatedByUserID: 1,
ProjectPath: projectDir,
})
}))
payload := customTemplateUpdatePayload{
Title: "Git Template",
Description: "Updated via git",
RepositoryURL: "https://github.com/example/repo",
RepositoryReferenceName: "refs/heads/main",
ComposeFilePathInRepository: filesystem.ComposeFileDefaultName,
Type: portainer.DockerComposeStack,
Platform: portainer.CustomTemplatePlatformLinux,
}
r := updateTemplateRequest(t, "1", payload, &security.RestrictedRequestContext{UserID: 1, IsAdmin: true})
rr := httptest.NewRecorder()
herr := handler.customTemplateUpdate(rr, r)
require.Nil(t, herr)
require.Equal(t, http.StatusOK, rr.Code)
var tmpl portainer.CustomTemplate
require.NoError(t, json.NewDecoder(rr.Body).Decode(&tmpl))
require.NotNil(t, tmpl.Artifact)
require.Len(t, tmpl.Artifact.Files, 1)
require.Equal(t, "deadbeef123", tmpl.Artifact.Files[0].Hash)
err := ds.ViewTx(func(tx dataservices.DataStoreTx) error {
stored, err := tx.CustomTemplate().Read(1)
require.NoError(t, err)
require.NotNil(t, stored.Artifact)
src, err := tx.Source().Read(stored.Artifact.Files[0].SourceID)
require.NoError(t, err)
require.Equal(t, portainer.SourceTypeGit, src.Type)
require.Equal(t, "https://github.com/example/repo", src.Git.URL)
return nil
})
require.NoError(t, err)
}

View File

@@ -4,11 +4,14 @@ import (
"net/http"
"sync"
"github.com/gorilla/mux"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/http/security"
"github.com/portainer/portainer/api/internal/authorization"
"github.com/portainer/portainer/api/slicesx"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/gorilla/mux"
)
// Handler is the HTTP handler used to handle environment(endpoint) group operations.
@@ -48,5 +51,27 @@ func NewHandler(bouncer security.BouncerService, dataStore dataservices.DataStor
}
func userCanEditTemplate(customTemplate *portainer.CustomTemplate, securityContext *security.RestrictedRequestContext) bool {
return securityContext.IsAdmin || customTemplate.CreatedByUserID == securityContext.UserID
resourceControl := customTemplate.ResourceControl
if securityContext.IsAdmin {
return true
}
if resourceControl == nil || resourceControl.AdministratorsOnly {
return false
}
if resourceControl.Public {
return true
}
if customTemplate.CreatedByUserID != securityContext.UserID {
return false
}
teamIDs := slicesx.Map(securityContext.UserMemberships, func(m portainer.TeamMembership) portainer.TeamID {
return m.TeamID
})
return authorization.UserCanAccessResource(securityContext.UserID, teamIDs, resourceControl)
}

View File

@@ -2,11 +2,49 @@ package customtemplates
import (
"errors"
"regexp"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
)
func validateVariablesDefinitions(variables []portainer.CustomTemplateVariableDefinition) error {
func populateGitConfig(tx dataservices.DataStoreTx, template *portainer.CustomTemplate) {
if template.Artifact == nil || len(template.Artifact.Files) == 0 {
return
}
file := template.Artifact.Files[0]
src, err := tx.Source().Read(file.SourceID)
if err != nil || src.Git == nil {
return
}
cfg := *src.Git
cfg.ReferenceName = file.Ref
cfg.ConfigFilePath = file.Path
cfg.ConfigHash = file.Hash
if cfg.Authentication != nil {
sanitized := *cfg.Authentication
sanitized.Password = ""
cfg.Authentication = &sanitized
}
template.GitConfig = &cfg
}
// IsValidNote reports whether note is safe to display. Notes containing <img> tags are rejected.
func IsValidNote(note string) bool {
if len(note) == 0 {
return true
}
match, _ := regexp.MatchString("<img", note)
return !match
}
// ValidateVariablesDefinitions returns an error if any variable definition is missing a required field.
func ValidateVariablesDefinitions(variables []portainer.CustomTemplateVariableDefinition) error {
for _, variable := range variables {
if variable.Name == "" {
return errors.New("variable name is required")

View File

@@ -0,0 +1,173 @@
package customtemplates
import (
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/datastore"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/stretchr/testify/require"
)
func TestPopulateGitConfig_NilArtifactIsNoOp(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
template := &portainer.CustomTemplate{ID: 1}
err := store.ViewTx(func(tx dataservices.DataStoreTx) error {
populateGitConfig(tx, template)
return nil
})
require.NoError(t, err)
require.Nil(t, template.GitConfig)
}
func TestPopulateGitConfig_EmptySourceIDsIsNoOp(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
template := &portainer.CustomTemplate{
ID: 1,
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{},
},
}
err := store.ViewTx(func(tx dataservices.DataStoreTx) error {
populateGitConfig(tx, template)
return nil
})
require.NoError(t, err)
require.Nil(t, template.GitConfig)
}
func TestPopulateGitConfig_SourceWithNilGitConfigIsNoOp(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
var srcID portainer.SourceID
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
src := &portainer.Source{Type: portainer.SourceTypeGit}
err := tx.Source().Create(src)
require.NoError(t, err)
srcID = src.ID
return nil
})
require.NoError(t, err)
template := &portainer.CustomTemplate{
ID: 1,
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{{SourceID: srcID}},
},
}
err = store.ViewTx(func(tx dataservices.DataStoreTx) error {
populateGitConfig(tx, template)
return nil
})
require.NoError(t, err)
require.Nil(t, template.GitConfig)
}
func TestPopulateGitConfig_PopulatesFromSourceAndArtifact(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
var srcID portainer.SourceID
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
src := &portainer.Source{
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: "https://github.com/example/repo",
TLSSkipVerify: true,
},
}
err := tx.Source().Create(src)
require.NoError(t, err)
srcID = src.ID
return nil
})
require.NoError(t, err)
template := &portainer.CustomTemplate{
ID: 1,
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{{
Ref: "refs/heads/main",
Path: "docker-compose.yml",
Hash: "abc123",
SourceID: srcID,
}},
},
}
err = store.ViewTx(func(tx dataservices.DataStoreTx) error {
populateGitConfig(tx, template)
return nil
})
require.NoError(t, err)
require.NotNil(t, template.GitConfig)
require.Equal(t, "https://github.com/example/repo", template.GitConfig.URL)
require.True(t, template.GitConfig.TLSSkipVerify)
require.Equal(t, "refs/heads/main", template.GitConfig.ReferenceName)
require.Equal(t, "docker-compose.yml", template.GitConfig.ConfigFilePath)
require.Equal(t, "abc123", template.GitConfig.ConfigHash)
}
func TestPopulateGitConfig_StripsPassword(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
var srcID portainer.SourceID
err := store.UpdateTx(func(tx dataservices.DataStoreTx) error {
src := &portainer.Source{
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: "https://github.com/example/repo",
Authentication: &gittypes.GitAuthentication{
Username: "user",
Password: "secret",
},
},
}
err := tx.Source().Create(src)
require.NoError(t, err)
srcID = src.ID
return nil
})
require.NoError(t, err)
template := &portainer.CustomTemplate{
ID: 1,
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{{SourceID: srcID}},
},
}
err = store.ViewTx(func(tx dataservices.DataStoreTx) error {
populateGitConfig(tx, template)
return nil
})
require.NoError(t, err)
require.NotNil(t, template.GitConfig)
require.NotNil(t, template.GitConfig.Authentication)
require.Equal(t, "user", template.GitConfig.Authentication.Username)
require.Empty(t, template.GitConfig.Authentication.Password)
}

View File

@@ -0,0 +1,95 @@
package docker
import (
"net/http"
"net/http/httptest"
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/apikey"
"github.com/portainer/portainer/api/datastore"
dockerdomain "github.com/portainer/portainer/api/docker"
dockerclient "github.com/portainer/portainer/api/docker/client"
"github.com/portainer/portainer/api/http/security"
"github.com/portainer/portainer/api/internal/authorization"
"github.com/portainer/portainer/api/internal/testhelpers"
"github.com/portainer/portainer/api/jwt"
"github.com/portainer/portainer/pkg/fips"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// unreachableDockerURL points the test environment at a port that refuses connections, so an
// authorized caller fails fast when the handler builds a docker client rather than blocking on a
// real daemon. The authorization middleware runs before this, which is what these tests assert.
const unreachableDockerURL = "tcp://127.0.0.1:1"
func newDashboardAuthTestHandler(t *testing.T) (*Handler, *jwt.Service, *datastore.Store) {
t.Helper()
fips.InitFIPS(false)
_, store := datastore.MustNewTestStore(t, true, true)
require.NoError(t, store.Endpoint().Create(&portainer.Endpoint{
ID: 1, Name: "docker-env", Type: portainer.DockerEnvironment, URL: unreachableDockerURL,
}))
jwtService, err := jwt.NewService("1h", store)
require.NoError(t, err)
bouncer := security.NewRequestBouncer(t.Context(), store, jwtService, apikey.NewAPIKeyService(nil, nil))
factory := dockerclient.NewClientFactory(nil, nil)
authorizationService := authorization.NewService(store)
containerService := dockerdomain.NewContainerService(factory, store)
handler := NewHandler(bouncer, authorizationService, store, factory, containerService)
return handler, jwtService, store
}
func dashboardRequest(t *testing.T, handler *Handler, jwtService *jwt.Service, user *portainer.User) *httptest.ResponseRecorder {
t.Helper()
req := httptest.NewRequest(http.MethodGet, "/docker/1/dashboard", nil)
tk, _, err := jwtService.GenerateToken(&portainer.TokenData{ID: user.ID, Username: user.Username, Role: user.Role})
require.NoError(t, err)
testhelpers.AddTestSecurityCookie(req, tk)
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
return rr
}
// TestEndpointAuthorization_DeniedUser_Returns403 verifies that the docker /dashboard
// route rejects users with no access policy for the target environment (R8S-1057).
func TestEndpointAuthorization_DeniedUser_Returns403(t *testing.T) {
handler, jwtService, store := newDashboardAuthTestHandler(t)
noAccessUser := &portainer.User{
Username: "no-access",
Role: portainer.StandardUserRole,
PortainerAuthorizations: authorization.DefaultPortainerAuthorizations(),
}
require.NoError(t, store.User().Create(noAccessUser))
// A standard user with no access policy must be rejected before the dashboard handler
// builds a docker client for the environment.
rr := dashboardRequest(t, handler, jwtService, noAccessUser)
assert.Equal(t, http.StatusForbidden, rr.Code)
}
// TestEndpointAuthorization_AuthorizedUser_NotForbidden verifies that the docker /dashboard
// route lets an authorized caller through the authorization middleware (R8S-1057). The request
// fails later when the handler cannot reach the docker daemon, but it must not be rejected with 403.
func TestEndpointAuthorization_AuthorizedUser_NotForbidden(t *testing.T) {
handler, jwtService, store := newDashboardAuthTestHandler(t)
adminUser := &portainer.User{Username: "admin", Role: portainer.AdministratorRole}
require.NoError(t, store.User().Create(adminUser))
rr := dashboardRequest(t, handler, jwtService, adminUser)
assert.NotEqual(t, http.StatusForbidden, rr.Code)
}

View File

@@ -44,7 +44,9 @@ func NewHandler(bouncer security.BouncerService, authorizationService *authoriza
endpointRouter.Use(bouncer.AuthenticatedAccess)
endpointRouter.Use(middlewares.WithEndpoint(dataStore.Endpoint(), "id"), dockerOnlyMiddleware)
endpointRouter.Handle("/dashboard", httperror.LoggerHandler(h.dashboard)).Methods(http.MethodGet)
// /dashboard is the only route on this router without its own endpoint authorization;
// the containers/images sub-routers already apply CheckEndpointAuthorization.
endpointRouter.Handle("/dashboard", middlewares.CheckEndpointAuthorization(bouncer)(httperror.LoggerHandler(h.dashboard))).Methods(http.MethodGet)
containersHandler := containers.NewHandler("/docker/{id}/containers", bouncer, dataStore, dockerClientFactory, containerService)
endpointRouter.PathPrefix("/containers").Handler(containersHandler)

View File

@@ -115,6 +115,11 @@ func (handler *Handler) createEdgeStackFromFileUpload(r *http.Request, tx datase
if dryrun {
return stack, nil
}
if err := stackutils.ValidateEdgeStackComposeContent(r.Context(), payload.DeploymentType, payload.StackFileContent); err != nil {
return nil, httperrors.NewInvalidPayloadError(err.Error())
}
stack.CreatedByUserId = fmt.Sprintf("%d", tokenData.ID)
stack.CreatedBy = stackutils.SanitizeLabel(tokenData.Username)

View File

@@ -9,10 +9,12 @@ import (
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/filesystem"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/gitops/sources"
httperrors "github.com/portainer/portainer/api/http/errors"
"github.com/portainer/portainer/api/stacks/stackutils"
"github.com/portainer/portainer/pkg/edge"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
"github.com/portainer/portainer/pkg/validate"
"github.com/pkg/errors"
@@ -25,15 +27,18 @@ type edgeStackFromGitRepositoryPayload struct {
// Name must start with a lowercase character or number
// Example: stack-name or stack_123 or stackName
Name string `example:"stack-name" validate:"required"`
// URL of a Git repository hosting the Stack file
RepositoryURL string `example:"https://github.com/openfaas/faas" validate:"required"`
// SourceID references an existing Source for git credentials/URL.
// When set, the inline URL and authentication fields are ignored.
SourceID portainer.SourceID `example:"1"`
// Deprecated: Use SourceID instead. URL of a Git repository hosting the Stack file.
RepositoryURL string `example:"https://github.com/openfaas/faas"`
// Reference name of a Git repository hosting the Stack file
RepositoryReferenceName string `example:"refs/heads/master"`
// Use basic authentication to clone the Git repository
// Deprecated: Use SourceID instead. Use basic authentication to clone the Git repository.
RepositoryAuthentication bool `example:"true"`
// Username used in basic authentication. Required when RepositoryAuthentication is true.
// Deprecated: Use SourceID instead. Username used in basic authentication.
RepositoryUsername string `example:"myGitUsername"`
// Password used in basic authentication. Required when RepositoryAuthentication is true.
// Deprecated: Use SourceID instead. Password used in basic authentication.
RepositoryPassword string `example:"myGitPassword"`
// Path to the Stack file inside the Git repository
FilePathInRepository string `example:"docker-compose.yml" default:"docker-compose.yml"`
@@ -48,7 +53,7 @@ type edgeStackFromGitRepositoryPayload struct {
Registries []portainer.RegistryID
// Uses the manifest's namespaces instead of the default one
UseManifestNamespaces bool
// TLSSkipVerify skips SSL verification when cloning the Git repository
// Deprecated: Use SourceID instead. TLSSkipVerify skips SSL verification when cloning the Git repository.
TLSSkipVerify bool `example:"false"`
}
@@ -61,12 +66,13 @@ func (payload *edgeStackFromGitRepositoryPayload) Validate(r *http.Request) erro
return httperrors.NewInvalidPayloadError("Invalid stack name. Stack name must only consist of lowercase alpha characters, numbers, hyphens, or underscores as well as start with a lowercase character or number")
}
if len(payload.RepositoryURL) == 0 || !validate.IsURL(payload.RepositoryURL) {
return httperrors.NewInvalidPayloadError("Invalid repository URL. Must correspond to a valid URL format")
}
if payload.RepositoryAuthentication && len(payload.RepositoryPassword) == 0 {
return httperrors.NewInvalidPayloadError("Invalid repository credentials. Password must be specified when authentication is enabled")
if payload.SourceID == 0 {
if len(payload.RepositoryURL) == 0 || !validate.IsURL(payload.RepositoryURL) {
return httperrors.NewInvalidPayloadError("Invalid repository URL. Must correspond to a valid URL format")
}
if payload.RepositoryAuthentication && len(payload.RepositoryPassword) == 0 {
return httperrors.NewInvalidPayloadError("Invalid repository credentials. Password must be specified when authentication is enabled")
}
}
if payload.DeploymentType != portainer.EdgeStackDeploymentCompose && payload.DeploymentType != portainer.EdgeStackDeploymentKubernetes {
@@ -118,18 +124,22 @@ func (handler *Handler) createEdgeStackFromGitRepository(r *http.Request, tx dat
return stack, nil
}
repoConfig := gittypes.RepoConfig{
URL: payload.RepositoryURL,
ReferenceName: payload.RepositoryReferenceName,
ConfigFilePath: payload.FilePathInRepository,
TLSSkipVerify: payload.TLSSkipVerify,
repoConfig, httpErr := sources.ResolveRepoConfig(tx, sources.RepoConfigInput{
SourceID: payload.SourceID,
ReferenceName: payload.RepositoryReferenceName,
ConfigFilePath: payload.FilePathInRepository,
RepositoryURL: payload.RepositoryURL,
TLSSkipVerify: payload.TLSSkipVerify,
RepositoryAuthentication: payload.RepositoryAuthentication,
Username: payload.RepositoryUsername,
Password: payload.RepositoryPassword,
})
if httpErr != nil {
return nil, httpErr
}
if payload.RepositoryAuthentication {
repoConfig.Authentication = &gittypes.GitAuthentication{
Username: payload.RepositoryUsername,
Password: payload.RepositoryPassword,
}
if err := ssrf.CheckURL(r.Context(), repoConfig.URL); err != nil {
return nil, errors.Wrap(err, "repository URL blocked by SSRF policy")
}
stack.CreatedByUserId = fmt.Sprintf("%d", tokenData.ID)

View File

@@ -93,6 +93,10 @@ func (handler *Handler) createEdgeStackFromFileContent(r *http.Request, tx datas
return stack, nil
}
if err := stackutils.ValidateEdgeStackComposeContent(r.Context(), payload.DeploymentType, []byte(payload.StackFileContent)); err != nil {
return nil, httperrors.NewInvalidPayloadError(err.Error())
}
return handler.edgeStacksService.PersistEdgeStack(tx, stack, func(stackFolder string, relatedEndpointIds []portainer.EndpointID) (composePath string, manifestPath string, projectPath string, err error) {
return handler.storeFileContent(tx, stackFolder, payload.DeploymentType, relatedEndpointIds, []byte(payload.StackFileContent))
})

View File

@@ -7,6 +7,7 @@ import (
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/internal/edge"
"github.com/portainer/portainer/api/set"
"github.com/portainer/portainer/api/stacks/stackutils"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
@@ -61,6 +62,10 @@ func (handler *Handler) edgeStackUpdate(w http.ResponseWriter, r *http.Request)
return httperror.BadRequest("Invalid request payload", err)
}
if err := stackutils.ValidateEdgeStackComposeContent(r.Context(), payload.DeploymentType, []byte(payload.StackFileContent)); err != nil {
return httperror.BadRequest("Stack file contains a URL blocked by the SSRF policy", err)
}
var stack *portainer.EdgeStack
if err := handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
stack, err = handler.updateEdgeStack(tx, portainer.EdgeStackID(stackID), payload)

View File

@@ -135,6 +135,10 @@ func (handler *Handler) parseHeaders(r *http.Request, endpoint *portainer.Endpoi
version := r.Header.Get(portainer.PortainerAgentHeader)
endpoint.Agent.Version = version
if gpuOperatorHeader := r.Header.Get(portainer.HTTPResponseAgentGPUOperator); gpuOperatorHeader != "" {
endpoint.Kubernetes.Flags.GPUOperator = gpuOperatorHeader == "true"
}
return nil
}

View File

@@ -4,6 +4,7 @@ import (
"net/http"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
@@ -18,7 +19,8 @@ import (
// @accept json
// @produce json
// @param id path int true "Environment(Endpoint) group identifier"
// @success 200 {object} portainer.EndpointGroup "Success"
// @param size query boolean false "If true, include the number of environments and breakdown by type"
// @success 200 {object} EndpointGroupResponse "Success"
// @failure 400 "Invalid request"
// @failure 404 "EndpointGroup not found"
// @failure 500 "Server error"
@@ -29,12 +31,42 @@ func (handler *Handler) endpointGroupInspect(w http.ResponseWriter, r *http.Requ
return httperror.BadRequest("Invalid environment group identifier route variable", err)
}
endpointGroup, err := handler.DataStore.EndpointGroup().Read(portainer.EndpointGroupID(endpointGroupID))
if handler.DataStore.IsErrObjectNotFound(err) {
return httperror.NotFound("Unable to find an environment group with the specified identifier inside the database", err)
} else if err != nil {
return httperror.InternalServerError("Unable to find an environment group with the specified identifier inside the database", err)
includeSize, err := request.RetrieveBooleanQueryParameter(r, "size", true)
if err != nil {
return httperror.BadRequest("Invalid query parameter: size", err)
}
return response.JSON(w, endpointGroup)
groupID := portainer.EndpointGroupID(endpointGroupID)
var endpointGroup *portainer.EndpointGroup
var endpoints []portainer.Endpoint
if err := handler.DataStore.ViewTx(func(tx dataservices.DataStoreTx) error {
var err error
endpointGroup, err = tx.EndpointGroup().Read(groupID)
if err != nil {
return err
}
if includeSize {
endpoints, err = tx.Endpoint().Endpoints()
}
return err
}); err != nil {
if handler.DataStore.IsErrObjectNotFound(err) {
return httperror.NotFound("Unable to find an environment group with the specified identifier inside the database", err)
}
return httperror.InternalServerError("Unable to retrieve environment group details", err)
}
resp := EndpointGroupResponse{
EndpointGroup: *endpointGroup,
}
if includeSize {
countMap, typeInfoMap := computeGroupSizeInfo([]portainer.EndpointGroup{*endpointGroup}, endpoints)
resp.Total = countMap[groupID]
resp.TypeInfo = typeInfoMap[groupID]
}
return response.JSON(w, resp)
}

View File

@@ -93,7 +93,7 @@ func (handler *Handler) endpointSummaryCounts(w http.ResponseWriter, r *http.Req
// write LastCheckInDate back to the database, so the tx value grows stale.
// The tx path cannot access the in-memory map; this non-tx access is
// intentional.
endpointSvc := handler.DataStore.Endpoint() //nolint:forbidigo
endpointSvc := handler.DataStore.Endpoint()
for i := range endpoints {
if t, ok := endpointSvc.Heartbeat(endpoints[i].ID); ok {
endpoints[i].LastCheckInDate = t

View File

@@ -6,10 +6,13 @@ import (
"fmt"
"net/http"
portainer "github.com/portainer/portainer/api"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/gitops/sources"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
"github.com/portainer/portainer/pkg/validate"
"github.com/rs/zerolog/log"
)
@@ -19,19 +22,32 @@ type fileResponse struct {
}
type repositoryFilePreviewPayload struct {
Repository string `json:"repository" example:"https://github.com/openfaas/faas" validate:"required"`
Reference string `json:"reference" example:"refs/heads/master"`
Username string `json:"username" example:"myGitUsername"`
Password string `json:"password" example:"myGitPassword"`
// SourceID resolves URL and auth from the stored Source record.
// When set, the inline Repository/Username/Password/TLSSkipVerify fields are ignored.
SourceID portainer.SourceID `json:"sourceID" example:"1"`
Reference string `json:"reference" example:"refs/heads/master"`
// Path to file whose content will be read
TargetFile string `json:"targetFile" example:"docker-compose.yml"`
// TLSSkipVerify skips SSL verification when cloning the Git repository
TLSSkipVerify bool `example:"false"`
// URL of a Git repository to preview.
// Deprecated: use SourceID instead
Repository string `json:"repository" example:"https://github.com/openfaas/faas"`
// Username for git authentication.
// Deprecated: use SourceID instead
Username string `json:"username" example:"myGitUsername"`
// Password for git authentication.
// Deprecated: use SourceID instead
Password string `json:"password" example:"myGitPassword"`
// TLSSkipVerify skips SSL verification when cloning the Git repository.
// Deprecated: use SourceID instead
TLSSkipVerify bool `json:"tlsSkipVerify" example:"false"`
}
func (payload *repositoryFilePreviewPayload) Validate(r *http.Request) error {
if len(payload.Repository) == 0 || !validate.IsURL(payload.Repository) {
return errors.New("invalid repository URL. Must correspond to a valid URL format")
if payload.SourceID == 0 {
if len(payload.Repository) == 0 || !validate.IsURL(payload.Repository) {
return errors.New("invalid repository URL. Must correspond to a valid URL format")
}
}
if len(payload.Reference) == 0 {
@@ -56,6 +72,7 @@ func (payload *repositoryFilePreviewPayload) Validate(r *http.Request) error {
// @param body body repositoryFilePreviewPayload true "Template details"
// @success 200 {object} fileResponse "Success"
// @failure 400 "Invalid request"
// @failure 404 "Source not found"
// @failure 500 "Server error"
// @router /gitops/repo/file/preview [post]
func (handler *Handler) gitOperationRepoFilePreview(w http.ResponseWriter, r *http.Request) *httperror.HandlerError {
@@ -65,6 +82,29 @@ func (handler *Handler) gitOperationRepoFilePreview(w http.ResponseWriter, r *ht
return httperror.BadRequest("Invalid request payload", err)
}
repoURL := payload.Repository
username := payload.Username
password := payload.Password
tlsSkipVerify := payload.TLSSkipVerify
if payload.SourceID != 0 {
src, httpErr := sources.ValidateGitSourceAccess(handler.dataStore, payload.SourceID)
if httpErr != nil {
return httpErr
}
repoURL = src.Git.URL
if src.Git.Authentication != nil {
username = src.Git.Authentication.Username
password = src.Git.Authentication.Password
}
tlsSkipVerify = src.Git.TLSSkipVerify
}
if err := ssrf.CheckURL(r.Context(), repoURL); err != nil {
return httperror.BadRequest("Repository URL blocked by SSRF policy", err)
}
projectPath, err := handler.fileService.GetTemporaryPath()
if err != nil {
return httperror.InternalServerError("Unable to create temporary folder", err)
@@ -73,11 +113,11 @@ func (handler *Handler) gitOperationRepoFilePreview(w http.ResponseWriter, r *ht
err = handler.gitService.CloneRepository(
context.TODO(),
projectPath,
payload.Repository,
repoURL,
payload.Reference,
payload.Username,
payload.Password,
payload.TLSSkipVerify,
username,
password,
tlsSkipVerify,
)
if err != nil {
if errors.Is(err, gittypes.ErrAuthenticationFailure) {

View File

@@ -8,17 +8,17 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/gitops/workflows"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
"github.com/portainer/portainer/pkg/validate"
)
// GitAuthenticationPayload holds authentication parameters for a git source
type GitAuthenticationPayload struct {
Username string `json:"username"`
Password string `json:"password"`
Provider gittypes.GitProvider `json:"provider"`
AuthorizationType gittypes.GitCredentialAuthType `json:"authorizationType"`
Username string `json:"username"`
Password string `json:"password"`
}
// GitSourceCreatePayload holds the parameters for creating a git-backed source
@@ -31,45 +31,17 @@ type GitSourceCreatePayload struct {
// Validate implements the portainer.Validatable interface
func (payload *GitSourceCreatePayload) Validate(_ *http.Request) error {
if strings.TrimSpace(payload.URL) == "" {
return errors.New("url is required")
if !validate.IsURL(payload.URL) {
return errors.New("invalid repository URL. Must correspond to a valid URL format")
}
return nil
}
// BuildGitSource constructs a portainer.Source from a GitSourceCreatePayload
func BuildGitSource(payload GitSourceCreatePayload) *portainer.Source {
gitConfig := &gittypes.RepoConfig{
URL: payload.URL,
TLSSkipVerify: payload.TLSSkipVerify,
}
if payload.Authentication != nil {
gitConfig.Authentication = &gittypes.GitAuthentication{
Username: payload.Authentication.Username,
Password: payload.Authentication.Password,
Provider: payload.Authentication.Provider,
AuthorizationType: payload.Authentication.AuthorizationType,
}
}
name := payload.Name
if strings.TrimSpace(name) == "" {
name = gittypes.RepoName(payload.URL)
}
return &portainer.Source{
Name: name,
Type: portainer.SourceTypeGit,
GitConfig: gitConfig,
}
}
// @id GitOpsSourcesCreateGit
// @summary Create a Git source
// @description Creates a new GitOps source backed by a Git repository.
// @description **Access policy**: admin
// @description **Access policy**: administrator
// @tags gitops
// @security ApiKeyAuth
// @security jwt
@@ -79,6 +51,7 @@ func BuildGitSource(payload GitSourceCreatePayload) *portainer.Source {
// @success 201 {object} portainer.Source
// @failure 400 "Invalid request payload"
// @failure 403 "Access denied"
// @failure 409 "A source with this URL and credentials already exists"
// @failure 500 "Server error"
// @router /gitops/sources/git [post]
func (h *Handler) gitSourceCreate(w http.ResponseWriter, r *http.Request) *httperror.HandlerError {
@@ -88,15 +61,71 @@ func (h *Handler) gitSourceCreate(w http.ResponseWriter, r *http.Request) *httpe
return httperror.BadRequest("Invalid request payload", err)
}
src := BuildGitSource(payload)
src, err := BuildGitSource(payload)
if err != nil {
return httperror.BadRequest("Invalid request payload", err)
}
username, password := "", ""
if payload.Authentication != nil {
username = payload.Authentication.Username
password = payload.Authentication.Password
}
if err := h.dataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
if isUnique, err := workflows.ValidateUniqueSource(tx, payload.URL, username, password, 0); err != nil {
return err
} else if !isUnique {
return ErrDuplicateSource
}
return tx.Source().Create(src)
}); err != nil {
}); errors.Is(err, ErrDuplicateSource) {
return httperror.Conflict("A source with this URL and credentials already exists", err)
} else if err != nil {
return httperror.InternalServerError("Unable to create source", err)
}
src.GitConfig = gittypes.SanitizeRepoConfig(src.GitConfig)
src.Git = gittypes.SanitizeRepoConfig(src.Git)
return response.JSONWithStatus(w, src, http.StatusCreated)
}
// BuildGitSource constructs a portainer.Source from a GitSourceCreatePayload
func BuildGitSource(payload GitSourceCreatePayload) (*portainer.Source, error) {
src := BuildBaseGitSource(payload)
src.Git.Authentication = BuildAuth(payload.Authentication)
return src, nil
}
// BuildBaseGitSource constructs the source skeleton (name, URL, TLS) without
// authentication.
func BuildBaseGitSource(payload GitSourceCreatePayload) *portainer.Source {
name := payload.Name
if strings.TrimSpace(name) == "" {
name = gittypes.RepoName(payload.URL)
}
return &portainer.Source{
Name: name,
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: payload.URL,
TLSSkipVerify: payload.TLSSkipVerify,
},
}
}
// BuildAuth constructs basic git authentication from the payload, returning nil
// when no authentication is provided.
func BuildAuth(payload *GitAuthenticationPayload) *gittypes.GitAuthentication {
if payload == nil {
return nil
}
return &gittypes.GitAuthentication{
Username: payload.Username,
Password: payload.Password,
}
}

View File

@@ -16,22 +16,24 @@ import (
func TestBuildGitSource_DerivesNameFromURL(t *testing.T) {
t.Parallel()
src := BuildGitSource(GitSourceCreatePayload{
src, err := BuildGitSource(GitSourceCreatePayload{
URL: "https://github.com/org/my-repo.git",
})
require.NoError(t, err)
require.Equal(t, "my-repo", src.Name)
require.Equal(t, portainer.SourceTypeGit, src.Type)
require.Nil(t, src.GitConfig.Authentication)
require.Nil(t, src.Git.Authentication)
}
func TestBuildGitSource_UsesExplicitName(t *testing.T) {
t.Parallel()
src := BuildGitSource(GitSourceCreatePayload{
src, err := BuildGitSource(GitSourceCreatePayload{
Name: "custom-name",
URL: "https://github.com/org/repo.git",
})
require.NoError(t, err)
require.Equal(t, "custom-name", src.Name)
}
@@ -39,17 +41,18 @@ func TestBuildGitSource_UsesExplicitName(t *testing.T) {
func TestBuildGitSource_WithAuthentication(t *testing.T) {
t.Parallel()
src := BuildGitSource(GitSourceCreatePayload{
src, err := BuildGitSource(GitSourceCreatePayload{
URL: "https://github.com/org/repo.git",
Authentication: &GitAuthenticationPayload{
Username: "alice",
Password: "secret",
},
})
require.NoError(t, err)
require.NotNil(t, src.GitConfig.Authentication)
require.Equal(t, "alice", src.GitConfig.Authentication.Username)
require.Equal(t, "secret", src.GitConfig.Authentication.Password)
require.NotNil(t, src.Git.Authentication)
require.Equal(t, "alice", src.Git.Authentication.Username)
require.Equal(t, "secret", src.Git.Authentication.Password)
}
func TestGitSourceCreatePayload_Validate_EmptyURL(t *testing.T) {
@@ -93,8 +96,8 @@ func TestGitSourceCreate_Success(t *testing.T) {
require.Equal(t, "my-source", src.Name)
require.Equal(t, portainer.SourceTypeGit, src.Type)
require.NotZero(t, src.ID)
require.NotNil(t, src.GitConfig)
require.Equal(t, "https://github.com/org/repo.git", src.GitConfig.URL)
require.NotNil(t, src.Git)
require.Equal(t, "https://github.com/org/repo.git", src.Git.URL)
}
func TestGitSourceCreate_SanitizesCredentials(t *testing.T) {
@@ -124,10 +127,10 @@ func TestGitSourceCreate_SanitizesCredentials(t *testing.T) {
var src portainer.Source
err = json.NewDecoder(rr.Body).Decode(&src)
require.NoError(t, err)
require.NotNil(t, src.GitConfig)
require.NotNil(t, src.GitConfig.Authentication)
require.Equal(t, "alice", src.GitConfig.Authentication.Username)
require.Empty(t, src.GitConfig.Authentication.Password)
require.NotNil(t, src.Git)
require.NotNil(t, src.Git.Authentication)
require.Equal(t, "alice", src.Git.Authentication.Username)
require.Empty(t, src.Git.Authentication.Password)
}
func TestGitSourceCreate_MissingURL(t *testing.T) {
@@ -149,6 +152,128 @@ func TestGitSourceCreate_MissingURL(t *testing.T) {
require.Equal(t, http.StatusBadRequest, rr.Code)
}
func TestGitSourceCreate_ConflictOnDuplicateURLAndCredentials(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.User().Create(&portainer.User{ID: 1, Role: portainer.AdministratorRole})
}))
h := newTestHandler(t, store)
body, err := json.Marshal(GitSourceCreatePayload{
URL: "https://github.com/org/repo.git",
Authentication: &GitAuthenticationPayload{
Username: "alice",
Password: "secret",
},
})
require.NoError(t, err)
rr := httptest.NewRecorder()
h.ServeHTTP(rr, buildCreateReq(t, 1, body))
require.Equal(t, http.StatusCreated, rr.Code)
rr = httptest.NewRecorder()
h.ServeHTTP(rr, buildCreateReq(t, 1, body))
require.Equal(t, http.StatusConflict, rr.Code)
}
func TestGitSourceCreate_AllowsDuplicateURLWithDifferentCredentials(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.User().Create(&portainer.User{ID: 1, Role: portainer.AdministratorRole})
}))
h := newTestHandler(t, store)
first, err := json.Marshal(GitSourceCreatePayload{
URL: "https://github.com/org/repo.git",
Authentication: &GitAuthenticationPayload{
Username: "alice",
Password: "secret",
},
})
require.NoError(t, err)
second, err := json.Marshal(GitSourceCreatePayload{
URL: "https://github.com/org/repo.git",
Authentication: &GitAuthenticationPayload{
Username: "bob",
Password: "other",
},
})
require.NoError(t, err)
rr := httptest.NewRecorder()
h.ServeHTTP(rr, buildCreateReq(t, 1, first))
require.Equal(t, http.StatusCreated, rr.Code)
rr = httptest.NewRecorder()
h.ServeHTTP(rr, buildCreateReq(t, 1, second))
require.Equal(t, http.StatusCreated, rr.Code)
}
func TestGitSourceCreate_ConflictOnDuplicateAuthlessSource(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.User().Create(&portainer.User{ID: 1, Role: portainer.AdministratorRole})
}))
h := newTestHandler(t, store)
body, err := json.Marshal(GitSourceCreatePayload{
URL: "https://github.com/org/repo.git",
})
require.NoError(t, err)
rr := httptest.NewRecorder()
h.ServeHTTP(rr, buildCreateReq(t, 1, body))
require.Equal(t, http.StatusCreated, rr.Code)
rr = httptest.NewRecorder()
h.ServeHTTP(rr, buildCreateReq(t, 1, body))
require.Equal(t, http.StatusConflict, rr.Code)
}
func TestGitSourceCreate_AllowsAuthlessAndAuthenticatedSameURL(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
return tx.User().Create(&portainer.User{ID: 1, Role: portainer.AdministratorRole})
}))
h := newTestHandler(t, store)
authless, err := json.Marshal(GitSourceCreatePayload{
URL: "https://github.com/org/repo.git",
})
require.NoError(t, err)
authenticated, err := json.Marshal(GitSourceCreatePayload{
URL: "https://github.com/org/repo.git",
Authentication: &GitAuthenticationPayload{
Username: "alice",
Password: "secret",
},
})
require.NoError(t, err)
rr := httptest.NewRecorder()
h.ServeHTTP(rr, buildCreateReq(t, 1, authless))
require.Equal(t, http.StatusCreated, rr.Code)
rr = httptest.NewRecorder()
h.ServeHTTP(rr, buildCreateReq(t, 1, authenticated))
require.Equal(t, http.StatusCreated, rr.Code)
}
func TestGitSourceCreate_MalformedJSON(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)

View File

@@ -13,11 +13,11 @@ import (
"github.com/portainer/portainer/pkg/libhttp/response"
)
var ErrSourceInUse = errors.New("source is used by one or more workflows")
var ErrSourceInUse = errors.New("source is used by one or more workflows or custom templates")
// @id GitOpsSourcesDelete
// @summary Delete a source
// @description Deletes an existing GitOps source. Returns 409 if the source is referenced by any workflow.
// @description Deletes an existing GitOps source. Returns 409 if the source is referenced by any workflow or custom template.
// @description **Access policy**: admin
// @tags gitops
// @security ApiKeyAuth
@@ -27,7 +27,7 @@ var ErrSourceInUse = errors.New("source is used by one or more workflows")
// @failure 400 "Invalid request"
// @failure 403 "Access denied"
// @failure 404 "Source not found"
// @failure 409 "Source is in use by one or more workflows"
// @failure 409 "Source is in use by one or more workflows or custom templates"
// @failure 500 "Server error"
// @router /gitops/sources/{id} [delete]
func (h *Handler) sourceDelete(w http.ResponseWriter, r *http.Request) *httperror.HandlerError {
@@ -49,18 +49,33 @@ func (h *Handler) sourceDelete(w http.ResponseWriter, r *http.Request) *httperro
}
for _, wf := range workflows {
if slices.ContainsFunc(wf.Artifacts, func(as portainer.ArtifactSources) bool {
return slices.Contains(as.SourceIDs, portainer.SourceID(sourceID))
if slices.ContainsFunc(wf.Artifacts, func(as portainer.Artifact) bool {
return slices.ContainsFunc(as.Files, func(f portainer.ArtifactFile) bool {
return f.SourceID == portainer.SourceID(sourceID)
})
}) {
return ErrSourceInUse
}
}
templates, err := tx.CustomTemplate().ReadAll(func(t portainer.CustomTemplate) bool {
return t.Artifact != nil && slices.ContainsFunc(t.Artifact.Files, func(f portainer.ArtifactFile) bool {
return f.SourceID == portainer.SourceID(sourceID)
})
})
if err != nil {
return err
}
if len(templates) > 0 {
return ErrSourceInUse
}
return tx.Source().Delete(portainer.SourceID(sourceID))
}); h.dataStore.IsErrObjectNotFound(err) {
return httperror.NotFound("Unable to find a source with the specified identifier", err)
} else if errors.Is(err, ErrSourceInUse) {
return httperror.Conflict("Source is used by one or more workflows", err)
return httperror.Conflict("Source is used by one or more workflows or custom templates", err)
} else if err != nil {
return httperror.InternalServerError("Unable to delete source", err)
}

View File

@@ -14,6 +14,7 @@ import (
func TestSourceDelete_Success(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
var srcID portainer.SourceID
@@ -35,6 +36,7 @@ func TestSourceDelete_Success(t *testing.T) {
func TestSourceDelete_NotFound(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
@@ -50,6 +52,7 @@ func TestSourceDelete_NotFound(t *testing.T) {
func TestSourceDelete_InUse(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
var srcID portainer.SourceID
@@ -59,7 +62,7 @@ func TestSourceDelete_InUse(t *testing.T) {
require.NoError(t, err)
srcID = src.ID
wf := &portainer.Workflow{Artifacts: []portainer.ArtifactSources{{SourceIDs: []portainer.SourceID{src.ID}}}}
wf := &portainer.Workflow{Artifacts: []portainer.Artifact{{Files: []portainer.ArtifactFile{{SourceID: src.ID}}}}}
err = tx.Workflow().Create(wf)
require.NoError(t, err)
@@ -75,6 +78,7 @@ func TestSourceDelete_InUse(t *testing.T) {
func TestSourceDelete_NonNumericID(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
@@ -87,3 +91,34 @@ func TestSourceDelete_NonNumericID(t *testing.T) {
require.Equal(t, http.StatusBadRequest, rr.Code)
}
func TestSourceDelete_InUseByCustomTemplate(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
var srcID portainer.SourceID
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
src := &portainer.Source{Name: "in-use-by-template", Type: portainer.SourceTypeGit}
err := tx.Source().Create(src)
require.NoError(t, err)
srcID = src.ID
ct := &portainer.CustomTemplate{
ID: 1,
Artifact: &portainer.Artifact{
Files: []portainer.ArtifactFile{{SourceID: src.ID}},
},
}
err = tx.CustomTemplate().Create(ct)
require.NoError(t, err)
return tx.User().Create(&portainer.User{ID: 1, Role: portainer.AdministratorRole})
}))
h := newTestHandler(t, store)
rr := httptest.NewRecorder()
h.ServeHTTP(rr, buildDeleteReq(t, 1, int(srcID)))
require.Equal(t, http.StatusConflict, rr.Code)
}

View File

@@ -13,8 +13,10 @@ import (
// FetchSourceWorkflows returns the workflows and stats for a single source.
func FetchSourceWorkflows(tx dataservices.DataStoreTx, src *portainer.Source) ([]ce.Workflow, ce.SourceStats, error) {
wfs, err := tx.Workflow().ReadAll(func(wf portainer.Workflow) bool {
return slices.ContainsFunc(wf.Artifacts, func(artifact portainer.ArtifactSources) bool {
return slices.Contains(artifact.SourceIDs, src.ID)
return slices.ContainsFunc(wf.Artifacts, func(artifact portainer.Artifact) bool {
return slices.ContainsFunc(artifact.Files, func(f portainer.ArtifactFile) bool {
return f.SourceID == src.ID
})
})
})
if err != nil {
@@ -40,7 +42,7 @@ func FetchSourceWorkflows(tx dataservices.DataStoreTx, src *portainer.Source) ([
stats := ce.SourceStats{EndpointIDs: set.Set[portainer.EndpointID]{}}
for _, stacks := range stacks {
items = append(items, ce.MapStackToWorkflow(stacks, src.GitConfig, unknown, unknown))
items = append(items, ce.MapStackToWorkflow(stacks, src.Git, unknown, unknown))
stats.WorkflowCount++
if stacks.EndpointID != 0 {
stats.EndpointIDs.Add(stacks.EndpointID)

View File

@@ -13,8 +13,7 @@ import (
)
type gitAuthInfo struct {
Type gittypes.GitCredentialAuthType `json:"type"`
Username string `json:"username"`
Username string `json:"username"`
}
type connectionInfo struct {
@@ -23,7 +22,7 @@ type connectionInfo struct {
Authentication *gitAuthInfo `json:"authentication,omitempty"`
}
type autoUpdateInfo struct {
type AutoUpdateInfo struct {
Mechanism string `json:"mechanism,omitempty"`
FetchInterval string `json:"fetchInterval,omitempty"`
}
@@ -32,14 +31,14 @@ type autoUpdateInfo struct {
type SourceDetail struct {
Source
Connection connectionInfo `json:"connection" validate:"required"`
AutoUpdate *autoUpdateInfo `json:"autoUpdate,omitempty"`
AutoUpdate *AutoUpdateInfo `json:"autoUpdate,omitempty"`
Workflows []workflows.Workflow `json:"workflows"`
}
// @id GitOpsSourceGet
// @summary Get a GitOps source by ID
// @description Returns a single GitOps source with its connection settings and linked workflows.
// @description **Access policy**: admin
// @description **Access policy**: authenticated
// @tags gitops
// @security ApiKeyAuth
// @security jwt
@@ -80,14 +79,14 @@ func (h *Handler) getSource(w http.ResponseWriter, r *http.Request) *httperror.H
return httperror.InternalServerError("Unable to retrieve source", err)
}
detail := BuildSourceDetail(h.buildSource(r.Context(), source, stats), source.GitConfig, sourceWfs)
detail := BuildSourceDetail(h.buildSource(r.Context(), source, stats), source.Git, sourceWfs)
return response.JSON(w, detail)
}
func BuildSourceDetail(baseSource Source, cfg *gittypes.RepoConfig, sourceWfs []workflows.Workflow) SourceDetail {
var autoUpdate *autoUpdateInfo
var autoUpdate *AutoUpdateInfo
if len(sourceWfs) > 0 {
autoUpdate = buildAutoUpdateInfo(sourceWfs[0].AutoUpdate)
autoUpdate = BuildAutoUpdateInfo(sourceWfs[0].AutoUpdate)
}
return SourceDetail{
@@ -114,24 +113,23 @@ func buildGitAuthInfo(auth *gittypes.GitAuthentication) *gitAuthInfo {
return nil
}
return &gitAuthInfo{
Type: auth.AuthorizationType,
Username: auth.Username,
}
}
func buildAutoUpdateInfo(autoUpdate *portainer.AutoUpdateSettings) *autoUpdateInfo {
func BuildAutoUpdateInfo(autoUpdate *portainer.AutoUpdateSettings) *AutoUpdateInfo {
if autoUpdate == nil {
return nil
}
switch {
case autoUpdate.Interval != "":
return &autoUpdateInfo{
return &AutoUpdateInfo{
Mechanism: "Interval",
FetchInterval: autoUpdate.Interval,
}
case autoUpdate.Webhook != "":
return &autoUpdateInfo{
return &AutoUpdateInfo{
Mechanism: "Webhook",
}
default:

View File

@@ -41,11 +41,12 @@ func NewHandler(bouncer security.BouncerService, dataStore dataservices.DataStor
authenticatedRouter.Use(bouncer.AuthenticatedAccess)
authenticatedRouter.Handle("", httperror.LoggerHandler(h.list)).Methods(http.MethodGet)
authenticatedRouter.Handle("/summary", httperror.LoggerHandler(h.summary)).Methods(http.MethodGet)
authenticatedRouter.Handle("/{id}", httperror.LoggerHandler(h.getSource)).Methods(http.MethodGet)
adminRouter := h.PathPrefix("/gitops/sources").Subrouter()
adminRouter.Use(bouncer.AdminAccess)
adminRouter.Handle("/git", httperror.LoggerHandler(h.gitSourceCreate)).Methods(http.MethodPost)
adminRouter.Handle("/{id}", httperror.LoggerHandler(h.getSource)).Methods(http.MethodGet)
adminRouter.Handle("/test", httperror.LoggerHandler(h.gitSourceTest)).Methods(http.MethodPost)
adminRouter.Handle("/{id}", httperror.LoggerHandler(h.gitSourceUpdate)).Methods(http.MethodPut)
adminRouter.Handle("/{id}", httperror.LoggerHandler(h.sourceDelete)).Methods(http.MethodDelete)
adminRouter.Handle("/{id}/test", httperror.LoggerHandler(h.sourceTestConnection)).Methods(http.MethodPost)

View File

@@ -23,20 +23,20 @@ func createGitWorkflow(t *testing.T, tx dataservices.DataStoreTx, stack *portain
t.Helper()
src := &portainer.Source{
Name: gittypes.RepoName(cfg.URL),
Type: portainer.SourceTypeGit,
GitConfig: cfg,
Name: gittypes.RepoName(cfg.URL),
Type: portainer.SourceTypeGit,
Git: cfg,
}
require.NoError(t, tx.Source().Create(src))
wf := &portainer.Workflow{
Artifacts: []portainer.ArtifactSources{{
Artifact: portainer.Artifact{
StackID: stack.ID,
ReferenceName: cfg.ReferenceName,
ConfigFilePath: cfg.ConfigFilePath,
},
SourceIDs: []portainer.SourceID{src.ID},
Artifacts: []portainer.Artifact{{
StackID: stack.ID,
Files: []portainer.ArtifactFile{{
SourceID: src.ID,
Path: cfg.ConfigFilePath,
Ref: cfg.ReferenceName,
}},
}},
}
require.NoError(t, tx.Workflow().Create(wf))

View File

@@ -19,12 +19,13 @@ func TestSourcesList_GroupsByURLAndCredentials(t *testing.T) {
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
cfg := gitCfg("https://github.com/org/repo")
src := &portainer.Source{Name: "repo", Type: portainer.SourceTypeGit, GitConfig: cfg}
src := &portainer.Source{Name: "repo", Type: portainer.SourceTypeGit, Git: cfg}
require.NoError(t, tx.Source().Create(src))
wfA := &portainer.Workflow{Artifacts: []portainer.ArtifactSources{{SourceIDs: []portainer.SourceID{src.ID}}}}
wfA := &portainer.Workflow{Artifacts: []portainer.Artifact{{Files: []portainer.ArtifactFile{{SourceID: src.ID}}}}}
require.NoError(t, tx.Workflow().Create(wfA))
wfB := &portainer.Workflow{Artifacts: []portainer.ArtifactSources{{SourceIDs: []portainer.SourceID{src.ID}}}}
wfB := &portainer.Workflow{Artifacts: []portainer.Artifact{{Files: []portainer.ArtifactFile{{SourceID: src.ID}}}}}
require.NoError(t, tx.Workflow().Create(wfB))
require.NoError(t, tx.Stack().Create(&portainer.Stack{ID: 1, Name: "stack-a", WorkflowID: wfA.ID}))

View File

@@ -14,10 +14,10 @@ import (
"github.com/portainer/portainer/pkg/libhttp/response"
)
// @id GitOpsSourcesTestGit
// @summary Test a Git source connection
// @id GitOpsSourcesTestById
// @summary Test the connection of a stored source
// @description Tests connectivity for a GitOps source, applying optional overrides to the stored configuration.
// @description **Access policy**: admin
// @description **Access policy**: administrator
// @tags gitops
// @security ApiKeyAuth
// @security jwt
@@ -58,11 +58,11 @@ func (h *Handler) sourceTestConnection(w http.ResponseWriter, r *http.Request) *
return httperror.InternalServerError("Unable to apply source changes", err)
}
if src.GitConfig == nil {
if src.Git == nil {
return httperror.InternalServerError("Source has no git configuration", nil)
}
result := testSourceConnection(r.Context(), h.gitService, src.GitConfig)
result := testSourceConnection(r.Context(), h.gitService, src.Git)
return response.JSON(w, result)
}
@@ -72,6 +72,40 @@ type ConnectionTestResult struct {
Error string `json:"error,omitempty"`
}
// @id GitOpsSourcesTest
// @summary Test a Git source connection
// @description Tests connectivity for Git connection details that have not been persisted yet.
// @description **Access policy**: administrator
// @tags gitops
// @security ApiKeyAuth
// @security jwt
// @accept json
// @produce json
// @param body body GitSourceCreatePayload true "Git connection details"
// @success 200 {object} ConnectionTestResult "Connection test result"
// @failure 400 "Invalid request payload"
// @failure 403 "Access denied"
// @failure 500 "Server error"
// @router /gitops/sources/test [post]
func (h *Handler) gitSourceTest(w http.ResponseWriter, r *http.Request) *httperror.HandlerError {
var payload GitSourceCreatePayload
if err := request.DecodeAndValidateJSONPayload(r, &payload); err != nil {
return httperror.BadRequest("Invalid request payload", err)
}
src, err := BuildGitSource(payload)
if err != nil {
return httperror.BadRequest("Invalid request payload", err)
}
if src.Git == nil {
return httperror.InternalServerError("Source has no git configuration", nil)
}
result := testSourceConnection(r.Context(), h.gitService, src.Git)
return response.JSON(w, result)
}
// testSourceConnection verifies that a git repository is reachable with the given config.
func testSourceConnection(ctx context.Context, gitService portainer.GitService, config *gittypes.RepoConfig) ConnectionTestResult {
var username, password string

View File

@@ -4,22 +4,20 @@ import (
"fmt"
portainer "github.com/portainer/portainer/api"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/api/gitops/workflows"
)
// Source represents a unique git repository used as a GitOps source across one or more workflows.
type Source struct {
ID portainer.SourceID `json:"id" validate:"required"`
Name string `json:"name" validate:"required"`
Type SourceType `json:"type" validate:"required"`
URL string `json:"url" validate:"required"`
Status workflows.Status `json:"status" validate:"required"`
Error string `json:"error,omitempty"`
Provider gittypes.GitProvider `json:"provider,omitempty"`
UsedBy int `json:"usedBy"`
Environments int `json:"environments"`
LastSync int64 `json:"lastSync"`
ID portainer.SourceID `json:"id" validate:"required"`
Name string `json:"name" validate:"required"`
Type SourceType `json:"type" validate:"required"`
URL string `json:"url" validate:"required"`
Status workflows.Status `json:"status" validate:"required"`
Error string `json:"error,omitempty"`
UsedBy int `json:"usedBy"`
Environments int `json:"environments"`
LastSync int64 `json:"lastSync"`
}
type SourceType string

View File

@@ -12,11 +12,12 @@ import (
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
"github.com/portainer/portainer/pkg/validate"
)
var (
ErrNotGitSource = errors.New("source is not a Git source")
ErrDuplicateSourceURL = errors.New("a source with this URL already exists")
ErrNotGitSource = errors.New("source is not a Git source")
ErrDuplicateSource = errors.New("a source with this URL and credentials already exists")
)
// GitSourceUpdatePayload holds the parameters for creating a git-backed source
@@ -29,21 +30,23 @@ type GitSourceUpdatePayload struct {
}
type GitAuthenticationUpdatePayload struct {
Username *string `json:"username"`
Password *string `json:"password"`
Provider *gittypes.GitProvider `json:"provider" swaggertype:"integer" enums:"0,1,2,3,4,5,6"`
AuthorizationType *gittypes.GitCredentialAuthType `json:"authorizationType" swaggertype:"integer" enums:"0,1"`
Username *string `json:"username"`
Password *string `json:"password"`
}
// Validate implements the portainer.Validatable interface
func (payload *GitSourceUpdatePayload) Validate(_ *http.Request) error {
if payload.URL != nil && !validate.IsURL(*payload.URL) {
return errors.New("invalid repository URL. Must correspond to a valid URL format")
}
return nil
}
// @id GitOpsSourcesUpdateGit
// @summary Update a Git source
// @description Updates an existing GitOps source backed by a Git repository.
// @description **Access policy**: admin
// @description **Access policy**: administrator
// @tags gitops
// @security ApiKeyAuth
// @security jwt
@@ -55,7 +58,7 @@ func (payload *GitSourceUpdatePayload) Validate(_ *http.Request) error {
// @failure 400 "Invalid request payload"
// @failure 403 "Access denied"
// @failure 404 "Source not found"
// @failure 409 "A source with this URL already exists"
// @failure 409 "A source with this URL and credentials already exists"
// @failure 500 "Server error"
// @router /gitops/sources/{id} [put]
func (h *Handler) gitSourceUpdate(w http.ResponseWriter, r *http.Request) *httperror.HandlerError {
@@ -77,14 +80,6 @@ func (h *Handler) gitSourceUpdate(w http.ResponseWriter, r *http.Request) *httpe
if err := h.dataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
var err error
if payload.URL != nil {
if isUnique, err := workflows.ValidateUniqueSourceURL(tx, *payload.URL, sourceID); err != nil {
return err
} else if !isUnique {
return ErrDuplicateSourceURL
}
}
if src, err = tx.Source().Read(sourceID); err != nil {
return err
}
@@ -93,24 +88,57 @@ func (h *Handler) gitSourceUpdate(w http.ResponseWriter, r *http.Request) *httpe
return err
}
username, password := "", ""
if src.Git != nil && src.Git.Authentication != nil {
username = src.Git.Authentication.Username
password = src.Git.Authentication.Password
}
if isUnique, err := workflows.ValidateUniqueSource(tx, src.Git.URL, username, password, sourceID); err != nil {
return err
} else if !isUnique {
return ErrDuplicateSource
}
return tx.Source().Update(src.ID, src)
}); h.dataStore.IsErrObjectNotFound(err) {
return httperror.NotFound("Unable to find a source with the specified identifier", err)
} else if errors.Is(err, ErrNotGitSource) {
return httperror.BadRequest("Source is not a Git source", err)
} else if errors.Is(err, ErrDuplicateSourceURL) {
return httperror.Conflict("A source with this URL already exists", err)
} else if errors.Is(err, ErrDuplicateSource) {
return httperror.Conflict("A source with this URL and credentials already exists", err)
} else if err != nil {
return httperror.InternalServerError("Unable to update source", err)
}
src.GitConfig = gittypes.SanitizeRepoConfig(src.GitConfig)
src.Git = gittypes.SanitizeRepoConfig(src.Git)
return response.JSON(w, src)
}
// ApplyGitSourceChanges applies the payload changes to the source in place
func ApplyGitSourceChanges(src *portainer.Source, payload GitSourceUpdatePayload) error {
if err := ApplyBaseGitSourceChanges(src, payload); err != nil {
return err
}
if payload.Authentication == nil {
return nil
}
if *payload.Authentication == (GitAuthenticationUpdatePayload{}) {
src.Git.Authentication = nil
return nil
}
src.Git.Authentication = ApplyAuthChanges(src.Git.Authentication, *payload.Authentication)
return nil
}
// ApplyBaseGitSourceChanges applies the non-authentication field changes (name,
// URL, reference, TLS) to the source in place, ensuring src.Git is set
func ApplyBaseGitSourceChanges(src *portainer.Source, payload GitSourceUpdatePayload) error {
if src.Type != portainer.SourceTypeGit {
return ErrNotGitSource
}
@@ -119,55 +147,41 @@ func ApplyGitSourceChanges(src *portainer.Source, payload GitSourceUpdatePayload
src.Name = *payload.Name
}
gitConfig := src.GitConfig
if gitConfig == nil {
gitConfig = &gittypes.RepoConfig{}
if src.Git == nil {
src.Git = &gittypes.RepoConfig{}
}
if payload.URL != nil {
gitConfig.URL = *payload.URL
src.Git.URL = *payload.URL
}
if payload.ReferenceName != nil {
gitConfig.ReferenceName = *payload.ReferenceName
src.Git.ReferenceName = *payload.ReferenceName
}
if payload.TLSSkipVerify != nil {
gitConfig.TLSSkipVerify = *payload.TLSSkipVerify
src.Git.TLSSkipVerify = *payload.TLSSkipVerify
}
var auth *gittypes.GitAuthentication
if payload.Authentication == nil {
auth = gitConfig.Authentication
} else if *payload.Authentication != (GitAuthenticationUpdatePayload{}) {
existing := gitConfig.Authentication
if existing != nil {
copied := *existing
auth = &copied
} else {
auth = &gittypes.GitAuthentication{}
}
authPayload := *payload.Authentication
if authPayload.AuthorizationType != nil {
auth.AuthorizationType = *authPayload.AuthorizationType
}
if authPayload.Username != nil {
auth.Username = *authPayload.Username
}
if authPayload.Password != nil {
auth.Password = *authPayload.Password
}
if authPayload.Provider != nil {
auth.Provider = *authPayload.Provider
}
}
gitConfig.Authentication = auth
src.GitConfig = gitConfig
return nil
}
// ApplyAuthChanges returns a copy of the existing authentication (or a fresh
// one) with the basic credential changes applied.
func ApplyAuthChanges(existing *gittypes.GitAuthentication, payload GitAuthenticationUpdatePayload) *gittypes.GitAuthentication {
auth := &gittypes.GitAuthentication{}
if existing != nil {
copied := *existing
auth = &copied
}
if payload.Username != nil {
auth.Username = *payload.Username
}
if payload.Password != nil {
auth.Password = *payload.Password
}
return auth
}

View File

@@ -45,8 +45,8 @@ func TestGitSourceUpdate_Success(t *testing.T) {
err = json.NewDecoder(rr.Body).Decode(&src)
require.NoError(t, err)
require.Equal(t, "new-name", src.Name)
require.NotNil(t, src.GitConfig)
require.Equal(t, "https://github.com/org/new.git", src.GitConfig.URL)
require.NotNil(t, src.Git)
require.Equal(t, "https://github.com/org/new.git", src.Git.URL)
}
func TestGitSourceUpdate_PreservesAuthWhenNotProvided(t *testing.T) {
@@ -58,7 +58,7 @@ func TestGitSourceUpdate_PreservesAuthWhenNotProvided(t *testing.T) {
src := &portainer.Source{
Name: "auth-source",
Type: portainer.SourceTypeGit,
GitConfig: &gittypes.RepoConfig{
Git: &gittypes.RepoConfig{
URL: "https://github.com/org/repo.git",
Authentication: &gittypes.GitAuthentication{
Username: "alice",
@@ -92,10 +92,10 @@ func TestGitSourceUpdate_PreservesAuthWhenNotProvided(t *testing.T) {
stored, err = tx.Source().Read(srcID)
return err
}))
require.NotNil(t, stored.GitConfig)
require.NotNil(t, stored.GitConfig.Authentication)
require.Equal(t, "alice", stored.GitConfig.Authentication.Username)
require.Equal(t, "secret", stored.GitConfig.Authentication.Password)
require.NotNil(t, stored.Git)
require.NotNil(t, stored.Git.Authentication)
require.Equal(t, "alice", stored.Git.Authentication.Username)
require.Equal(t, "secret", stored.Git.Authentication.Password)
}
func TestGitSourceUpdate_ClearsAuthWhenRequested(t *testing.T) {
@@ -107,7 +107,7 @@ func TestGitSourceUpdate_ClearsAuthWhenRequested(t *testing.T) {
src := &portainer.Source{
Name: "auth-source",
Type: portainer.SourceTypeGit,
GitConfig: &gittypes.RepoConfig{
Git: &gittypes.RepoConfig{
URL: "https://github.com/org/repo.git",
Authentication: &gittypes.GitAuthentication{
Username: "alice",
@@ -141,8 +141,8 @@ func TestGitSourceUpdate_ClearsAuthWhenRequested(t *testing.T) {
stored, err = tx.Source().Read(srcID)
return err
}))
require.NotNil(t, stored.GitConfig)
require.Nil(t, stored.GitConfig.Authentication)
require.NotNil(t, stored.Git)
require.Nil(t, stored.Git.Authentication)
}
func TestGitSourceUpdate_ReplacesAuthWhenProvided(t *testing.T) {
@@ -154,7 +154,7 @@ func TestGitSourceUpdate_ReplacesAuthWhenProvided(t *testing.T) {
src := &portainer.Source{
Name: "auth-source",
Type: portainer.SourceTypeGit,
GitConfig: &gittypes.RepoConfig{
Git: &gittypes.RepoConfig{
URL: "https://github.com/org/repo.git",
Authentication: &gittypes.GitAuthentication{
Username: "alice",
@@ -191,10 +191,10 @@ func TestGitSourceUpdate_ReplacesAuthWhenProvided(t *testing.T) {
stored, err = tx.Source().Read(srcID)
return err
}))
require.NotNil(t, stored.GitConfig)
require.NotNil(t, stored.GitConfig.Authentication)
require.Equal(t, "bob", stored.GitConfig.Authentication.Username)
require.Equal(t, "new-secret", stored.GitConfig.Authentication.Password)
require.NotNil(t, stored.Git)
require.NotNil(t, stored.Git.Authentication)
require.Equal(t, "bob", stored.Git.Authentication.Username)
require.Equal(t, "new-secret", stored.Git.Authentication.Password)
}
func TestGitSourceUpdate_NotFound(t *testing.T) {
@@ -225,7 +225,7 @@ func TestGitSourceUpdate_ConflictOnDuplicateURL(t *testing.T) {
existing := &portainer.Source{
Name: "existing",
Type: portainer.SourceTypeGit,
GitConfig: &gittypes.RepoConfig{
Git: &gittypes.RepoConfig{
URL: "https://github.com/org/existing.git",
},
}
@@ -300,6 +300,57 @@ func TestGitSourceUpdate_MalformedJSON(t *testing.T) {
require.Equal(t, http.StatusBadRequest, rr.Code)
}
func TestGitSourceUpdate_ConflictWhenAuthChangesMatchAnotherSource(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)
var srcID portainer.SourceID
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
existing := &portainer.Source{
Name: "existing",
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{
URL: "https://github.com/org/repo.git",
Authentication: &gittypes.GitAuthentication{
Username: "alice",
Password: "secret",
},
},
}
if err := tx.Source().Create(existing); err != nil {
return err
}
other := &portainer.Source{
Name: "other",
Type: portainer.SourceTypeGit,
Git: &gittypes.RepoConfig{URL: "https://github.com/org/repo.git"},
}
if err := tx.Source().Create(other); err != nil {
return err
}
srcID = other.ID
return tx.User().Create(&portainer.User{ID: 1, Role: portainer.AdministratorRole})
}))
h := newTestHandler(t, store)
alice := "alice"
secret := "secret"
body, err := json.Marshal(GitSourceUpdatePayload{
Authentication: &GitAuthenticationUpdatePayload{
Username: &alice,
Password: &secret,
},
})
require.NoError(t, err)
rr := httptest.NewRecorder()
h.ServeHTTP(rr, buildUpdateReq(t, 1, int(srcID), body))
require.Equal(t, http.StatusConflict, rr.Code)
}
func TestGitSourceUpdate_NonNumericID(t *testing.T) {
t.Parallel()
_, store := datastore.MustNewTestStore(t, false, true)

View File

@@ -11,8 +11,8 @@ import (
func (h *Handler) buildSource(ctx context.Context, src *portainer.Source, stats ce.SourceStats) Source {
var status ce.Status
var sourceErr string
if src.GitConfig != nil {
phase, _ := ce.ComputeGitPhasesForConfig(ctx, h.gitService, src.GitConfig)
if src.Git != nil {
phase, _ := ce.ComputeGitPhasesForConfig(ctx, h.gitService, src.Git)
status = phase.Status
sourceErr = phase.Error
} else {
@@ -20,12 +20,8 @@ func (h *Handler) buildSource(ctx context.Context, src *portainer.Source, stats
}
url := ""
var provider gittypes.GitProvider
if src.GitConfig != nil {
url = gittypes.SanitizeURL(src.GitConfig.URL)
if src.GitConfig.Authentication != nil {
provider = src.GitConfig.Authentication.Provider
}
if src.Git != nil {
url = gittypes.SanitizeURL(src.Git.URL)
}
return Source{
@@ -35,7 +31,6 @@ func (h *Handler) buildSource(ctx context.Context, src *portainer.Source, stats
URL: url,
Status: status,
Error: sourceErr,
Provider: provider,
UsedBy: stats.WorkflowCount,
Environments: len(stats.EndpointIDs),
LastSync: stats.LastSync,

View File

@@ -49,15 +49,15 @@ func TestRedactWorkflowCredentials(t *testing.T) {
func TestBuildAutoUpdateInfo(t *testing.T) {
t.Parallel()
assert.Nil(t, buildAutoUpdateInfo(nil))
assert.Nil(t, buildAutoUpdateInfo(&portainer.AutoUpdateSettings{}))
assert.Nil(t, BuildAutoUpdateInfo(nil))
assert.Nil(t, BuildAutoUpdateInfo(&portainer.AutoUpdateSettings{}))
got := buildAutoUpdateInfo(&portainer.AutoUpdateSettings{Interval: "5m"})
got := BuildAutoUpdateInfo(&portainer.AutoUpdateSettings{Interval: "5m"})
require.NotNil(t, got)
assert.Equal(t, "Interval", got.Mechanism)
assert.Equal(t, "5m", got.FetchInterval)
got = buildAutoUpdateInfo(&portainer.AutoUpdateSettings{Webhook: "abc123"})
got = BuildAutoUpdateInfo(&portainer.AutoUpdateSettings{Webhook: "abc123"})
require.NotNil(t, got)
assert.Equal(t, "Webhook", got.Mechanism)
assert.Empty(t, got.FetchInterval)

View File

@@ -47,17 +47,17 @@ func createGitStack(t *testing.T, tx dataservices.DataStoreTx, stack *portainer.
t.Helper()
if stack.GitConfig != nil {
src := &portainer.Source{GitConfig: stack.GitConfig, Type: portainer.SourceTypeGit}
src := &portainer.Source{Git: stack.GitConfig, Type: portainer.SourceTypeGit}
require.NoError(t, tx.Source().Create(src))
wf := &portainer.Workflow{Artifacts: []portainer.ArtifactSources{{
Artifact: portainer.Artifact{
StackID: stack.ID,
ReferenceName: stack.GitConfig.ReferenceName,
ConfigFilePath: stack.GitConfig.ConfigFilePath,
ConfigHash: stack.GitConfig.ConfigHash,
},
SourceIDs: []portainer.SourceID{src.ID},
wf := &portainer.Workflow{Artifacts: []portainer.Artifact{{
StackID: stack.ID,
Files: []portainer.ArtifactFile{{
SourceID: src.ID,
Path: stack.GitConfig.ConfigFilePath,
Ref: stack.GitConfig.ReferenceName,
Hash: stack.GitConfig.ConfigHash,
}},
}}}
require.NoError(t, tx.Workflow().Create(wf))

View File

@@ -16,6 +16,7 @@ import (
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
"github.com/portainer/portainer/pkg/libhttp/ssrf"
"github.com/rs/zerolog/log"
"github.com/pkg/errors"
@@ -65,6 +66,10 @@ func (handler *Handler) helmInstall(w http.ResponseWriter, r *http.Request) *htt
return httperror.BadRequest("Invalid Helm install payload", err)
}
if err := ssrf.CheckURL(r.Context(), payload.Repo); err != nil {
return httperror.BadRequest("Repository URL blocked by SSRF policy", err)
}
release, err := handler.installChart(r, payload, dryRun)
if err != nil {
return httperror.InternalServerError("Unable to install a chart", err)

Some files were not shown because too many files have changed in this diff Show More