Compare commits

..

48 Commits

Author SHA1 Message Date
Steven Kang cb84cb73ab chore: version bump 2.33.3 (#1351) 2025-10-30 11:56:33 +13:00
andres-portainer 941e86563a fix(CVE-2025-62725): upgrade github.com/docker/compose/v2 to v2.40.2 BE-12352 (#1344) 2025-10-29 18:17:39 -03:00
Malcolm Lockyer f72d6b97d3 fix(agent): for iamra and ecr login, detect errors and retry [be-12284] (#1309) 2025-10-29 17:24:02 +13:00
Steven Kang 32926aa8bf fix: add web socket headers for kubeconfig based access - 2.33.3 [r8s-592] (#1329) 2025-10-22 09:44:46 +13:00
Steven Kang 1849c61c38 fix: display dependency version for kubectl and helm - 2.33.3 [R8S-501] (#1282) 2025-10-07 16:23:43 +13:00
andres-portainer fd6d74602c feat(boltdb): attempt to compact using a read-only database BE-12287 (#1268) 2025-09-30 19:10:16 -03:00
Oscar Zhou 74b1dd04d1 fix(k8s): memory leak during k8s stack deployment [BE-12281] (#1264) 2025-09-30 18:00:12 +13:00
Steven Kang 7450501b7a chore: version bump 2.33.2 (#1257) 2025-09-25 14:29:28 +12:00
andres-portainer dcfe2d9809 feat(database): add a flag to compact on startup BE-12283 (#1256) 2025-09-24 18:43:54 -03:00
Ali c21c91632f fix(rbac): redirect on unauthorized namespace [r8s-564] (#1246)
Merging because this PR doesn't introduce any CI failures, compared to the release 2.33 CI run https://github.com/portainer/portainer-suite/actions/runs/17957775674
2025-09-24 13:22:42 +12:00
andres-portainer 732337615e fix(edgestacks): add a missing webhook uniqueness check BE-12219 (#1251) 2025-09-23 17:20:25 -03:00
LP B 6ea16c0060 fix(api/endpoints): edge stack status type filter no longer always include Pending envs (#1230) 2025-09-22 16:10:46 +02:00
Ali 4e7d4b60a5 fix(cve): fix frontend CVEs [r8s-563] (#1238) 2025-09-22 10:17:12 +12:00
Oscar Zhou 19e1cc2fbd fix(activitylog): remove export limit and fix search function [BE-12270] (#1232) 2025-09-19 14:45:14 +12:00
andres-portainer 68b9fef3f0 fix(kubernetes/cli): fix a data-race BE-12259 (#1227) 2025-09-18 10:22:29 -03:00
Viktor Pettersson 1e47df6611 chore(go): upgrade Go to 1.24.6 BE-12263 (#1220) 2025-09-18 11:44:09 +12:00
Oscar Zhou 405ce8f671 feat(edge): add option to allow always clone git repository [BE-12240] (#1207) 2025-09-17 18:25:47 +12:00
andres-portainer e9d31b3b7b fix(csp): update the Content-Security-Policy header BE-12228 (#1202) 2025-09-15 10:47:57 -03:00
LP B f97adc94ad fix(api/custom-templates): UAC-allowed users cannot fetch custom template details (#1199) 2025-09-12 15:38:58 +02:00
Malcolm Lockyer 11d6341765 fix(encryption): set correct default secret key path release [r8s-555] (#1184)
Co-authored-by: Gorbasch <57012534+mbegerau@users.noreply.github.com>
2025-09-11 16:32:52 +12:00
andres-portainer c3cf46b0e0 fix(auth): remove a nil pointer dereference BE-12149 (#1174) 2025-09-10 21:55:19 -03:00
andres-portainer ff746beba1 fix(csp): add google.com to the CSP header BE-12228 (#1176) 2025-09-10 15:01:00 -03:00
LP B da1672fc17 fix(api): standard users cannot connect or disconnect containers to networks (#1166) 2025-09-09 22:07:24 +02:00
Ali 7a9376cbaf fix(helm): update helm repo validation to match helm cli [r8s-531] (#1142) 2025-09-08 08:55:57 +12:00
Malcolm Lockyer c0f6410d80 fix(fips): encrypt the chisel private key file for fips [be-12132] (#1149) 2025-09-05 13:17:23 +12:00
andres-portainer 4b9ab98fd2 fix(git): add a minimum interval validation BE-12220 (#1145) 2025-09-04 15:11:24 -03:00
andres-portainer 3354ee4e4b fix(registries): clear sensitive fields in the update handler BE-12215 (#1129) 2025-09-03 10:41:27 -03:00
Steven Kang af3c45bea0 chore: version bump 2.33.1 (#1108) 2025-08-27 10:45:29 +12:00
andres-portainer 816a6f9bef chore(bbolt): upgrade bbolt to v1.4.3 BE-12193 (#1104) 2025-08-25 17:59:33 -03:00
Devon Steenberg e86ea22900 fix(sslflags): Deprecate ssl flags [BE-12168] (#1076) 2025-08-25 20:25:07 +12:00
Malcolm Lockyer 12b2acbc00 fix(standard): manual endpoint refresh fails to save new status [be-12188] (#1096) 2025-08-25 13:49:04 +12:00
Ali 4a8b42928e fix(environments): create k8s specific edge agent before connecting [r8s-438] (#1086)
Merging because this change is unrelated to the failing kubernetes/tests/helm-oci.spec.ts tests
2025-08-25 09:32:16 +12:00
Oscar Zhou 2e828b39da fix(autoupdate): update tooltips in edge stack gitops update [BE-12177] (#1080) 2025-08-23 10:55:57 +12:00
Steven Kang 49c6521c23 fix: GHSA-2464-8j7c-4cjm - release 2.33 [R8S-495] (#1089) 2025-08-22 14:03:16 +12:00
Steven Kang debf1a742b chore: version bump 2.33.0 (#1065) 2025-08-20 11:28:05 +12:00
James Player 5d3708ec3e fix(UI): add experimental features back in [r8s-483] (#1060) 2025-08-19 17:07:27 +12:00
Steven Kang 9320fd4c50 fix: cve-2025-55198 and cve-2025-55199 - release 2.33 [R8S-482] (#1058) 2025-08-19 16:22:54 +12:00
Steven Kang 974682bd98 chore: version bump to 2.33.0-rc2 (#1054) 2025-08-19 11:04:56 +12:00
Ali 631f1deb2e fix(helm): support http and custom tls helm registries, give help when misconfigured [r8s-472] (#1032)
Co-authored-by: JamesPlayer <james.player@portainer.io>
2025-08-18 12:07:41 +12:00
LP B 4169b045fb fix(api/edge-stacks): avoid overriding updates with old values (#1048) 2025-08-16 03:52:21 +02:00
andres-portainer 0a2a786aa3 fix(migrator): rewrite a migration so it is idempotent BE-12053 (#1043) 2025-08-15 09:18:31 -03:00
James Player 808f87206e fix(ui): Fixed react-select TooManyResultsSelector filter and improved scrolling (#1028) 2025-08-15 15:33:43 +12:00
Cara Ryan ed6fa82904 fix(pending-actions): Small improvements to pending actions (R8S-350) (#1025) 2025-08-15 10:51:45 +12:00
andres-portainer 9fc301110b fix(crypto): replace fips140 calls with fips calls BE-11979 (#1035) 2025-08-14 19:36:05 -03:00
andres-portainer 69101ac89a feat(openai): remove OpenAI BE-12018 (#1034) 2025-08-14 19:35:43 -03:00
Malcolm Lockyer 69d33dd432 fix(fips): use standard lib pbkdf2 [be-12164] (#1037) 2025-08-15 09:45:49 +12:00
Ali 389cbf748c fix(logs): improve log rendering performance [r8s-437] (#1023)
Merging because the same tests are failing in CE develop https://github.com/portainer/system-tests/actions/runs/16953578581
2025-08-14 13:53:35 +12:00
LP B d01b31f707 feat(api): Permissions-Policy header deny all (#1022) 2025-08-13 22:07:52 +02:00
310 changed files with 3753 additions and 6180 deletions
+1 -1
View File
@@ -6,7 +6,7 @@ body:
Thanks for suggesting an idea for Portainer!
Before opening a new idea or feature request, make sure that we do not have any duplicates already open. You can ensure this by [searching this discussion category](https://github.com/orgs/portainer/discussions/categories/ideas). If there is a duplicate, please add a comment to the existing idea instead.
Before opening a new idea or feature request, make sure that we do not have any duplicates already open. You can ensure this by [searching this discussion cagetory](https://github.com/orgs/portainer/discussions/categories/ideas). If there is a duplicate, please add a comment to the existing idea instead.
Also, be sure to check our [knowledge base](https://portal.portainer.io/knowledge) and [documentation](https://docs.portainer.io) as they may point you toward a solution.
-3
View File
@@ -94,9 +94,6 @@ body:
description: We only provide support for current versions of Portainer as per the lifecycle policy linked above. If you are on an older version of Portainer we recommend [updating first](https://docs.portainer.io/start/upgrade) in case your bug has already been fixed.
multiple: false
options:
- '2.34.0'
- '2.33.1'
- '2.33.0'
- '2.32.0'
- '2.31.3'
- '2.31.2'
-11
View File
@@ -1,11 +0,0 @@
version: "2"
linters:
default: none
enable:
- forbidigo
settings:
forbidigo:
forbid:
- pattern: ^dataservices.DataStore.(EdgeGroup|EdgeJob|EdgeStack|EndpointRelation|Endpoint|GitCredential|Registry|ResourceControl|Role|Settings|Snapshot|Stack|Tag|User)$
msg: Use a transaction instead
analyze-types: true
-14
View File
@@ -13,12 +13,6 @@ linters:
- perfsprint
- staticcheck
- unused
- mirror
- durationcheck
- errorlint
- govet
- zerologlint
- testifylint
settings:
staticcheck:
checks: ["all", "-ST1003", "-ST1005", "-ST1016", "-SA1019", "-QF1003"]
@@ -38,20 +32,12 @@ linters:
desc: use github.com/portainer/portainer/pkg/libcrypto
- pkg: github.com/portainer/libhttp
desc: use github.com/portainer/portainer/pkg/libhttp
- pkg: golang.org/x/crypto
desc: golang.org/x/crypto is not allowed because of FIPS mode
- pkg: github.com/ProtonMail/go-crypto/openpgp
desc: github.com/ProtonMail/go-crypto/openpgp is not allowed because of FIPS mode
forbidigo:
forbid:
- pattern: ^tls\.Config$
msg: Use crypto.CreateTLSConfiguration() instead
- pattern: ^tls\.Config\.(InsecureSkipVerify|MinVersion|MaxVersion|CipherSuites|CurvePreferences)$
msg: Do not set this field directly, use crypto.CreateTLSConfiguration() instead
- pattern: ^object\.(Commit|Tag)\.Verify$
msg: "Not allowed because of FIPS mode"
- pattern: ^(types\.SystemContext\.)?(DockerDaemonInsecureSkipTLSVerify|DockerInsecureSkipTLSVerify|OCIInsecureSkipTLSVerify)$
msg: "Not allowed because of FIPS mode"
analyze-types: true
exclusions:
generated: lax
+5 -2
View File
@@ -54,12 +54,14 @@ client-deps: ## Install client dependencies
tidy: ## Tidy up the go.mod file
@go mod tidy
##@ Cleanup
.PHONY: clean
clean: ## Remove all build and download artifacts
@echo "Clearing the dist directory..."
@rm -rf dist/*
##@ Testing
.PHONY: test test-client test-server
test: test-server test-client ## Run all tests
@@ -103,15 +105,16 @@ lint: lint-client lint-server ## Lint all code
lint-client: ## Lint client code
yarn lint
lint-server: tidy ## Lint server code
lint-server: ## Lint server code
golangci-lint run --timeout=10m -c .golangci.yaml
golangci-lint run --timeout=10m --new-from-rev=HEAD~ -c .golangci-forward.yaml
##@ Extension
.PHONY: dev-extension
dev-extension: build-server build-client ## Run the extension in development mode
make local -f build/docker-extension/Makefile
##@ Docs
.PHONY: docs-build docs-validate docs-clean docs-validate-clean
docs-build: init-dist ## Build docs
+12 -12
View File
@@ -10,31 +10,31 @@ func Test_generateRandomKey(t *testing.T) {
is := assert.New(t)
tests := []struct {
name string
wantLength int
name string
wantLenth int
}{
{
name: "Generate a random key of length 16",
wantLength: 16,
name: "Generate a random key of length 16",
wantLenth: 16,
},
{
name: "Generate a random key of length 32",
wantLength: 32,
name: "Generate a random key of length 32",
wantLenth: 32,
},
{
name: "Generate a random key of length 64",
wantLength: 64,
name: "Generate a random key of length 64",
wantLenth: 64,
},
{
name: "Generate a random key of length 128",
wantLength: 128,
name: "Generate a random key of length 128",
wantLenth: 128,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := GenerateRandomKey(tt.wantLength)
is.Len(got, tt.wantLength)
got := GenerateRandomKey(tt.wantLenth)
is.Equal(tt.wantLenth, len(got))
})
}
+30 -31
View File
@@ -10,10 +10,9 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/datastore"
"github.com/stretchr/testify/assert"
"github.com/rs/zerolog/log"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_SatisfiesAPIKeyServiceInterface(t *testing.T) {
@@ -31,7 +30,7 @@ func Test_GenerateApiKey(t *testing.T) {
t.Run("Successfully generates API key", func(t *testing.T) {
desc := "test-1"
rawKey, apiKey, err := service.GenerateApiKey(portainer.User{ID: 1}, desc)
require.NoError(t, err)
is.NoError(err)
is.NotEmpty(rawKey)
is.NotEmpty(apiKey)
is.Equal(desc, apiKey.Description)
@@ -39,7 +38,7 @@ func Test_GenerateApiKey(t *testing.T) {
t.Run("Api key prefix is 7 chars", func(t *testing.T) {
rawKey, apiKey, err := service.GenerateApiKey(portainer.User{ID: 1}, "test-2")
require.NoError(t, err)
is.NoError(err)
is.Equal(rawKey[:7], apiKey.Prefix)
is.Len(apiKey.Prefix, 7)
@@ -47,7 +46,7 @@ func Test_GenerateApiKey(t *testing.T) {
t.Run("Api key has 'ptr_' as prefix", func(t *testing.T) {
rawKey, _, err := service.GenerateApiKey(portainer.User{ID: 1}, "test-x")
require.NoError(t, err)
is.NoError(err)
is.Equal(portainerAPIKeyPrefix, "ptr_")
is.True(strings.HasPrefix(rawKey, "ptr_"))
@@ -56,7 +55,7 @@ func Test_GenerateApiKey(t *testing.T) {
t.Run("Successfully caches API key", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-3")
require.NoError(t, err)
is.NoError(err)
userFromCache, apiKeyFromCache, ok := service.cache.Get(apiKey.Digest)
is.True(ok)
@@ -66,7 +65,7 @@ func Test_GenerateApiKey(t *testing.T) {
t.Run("Decoded raw api-key digest matches generated digest", func(t *testing.T) {
rawKey, apiKey, err := service.GenerateApiKey(portainer.User{ID: 1}, "test-4")
require.NoError(t, err)
is.NoError(err)
generatedDigest := sha256.Sum256([]byte(rawKey))
@@ -84,10 +83,10 @@ func Test_GetAPIKey(t *testing.T) {
t.Run("Successfully returns all API keys", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-1")
require.NoError(t, err)
is.NoError(err)
apiKeyGot, err := service.GetAPIKey(apiKey.ID)
require.NoError(t, err)
is.NoError(err)
is.Equal(apiKey, apiKeyGot)
})
@@ -103,12 +102,12 @@ func Test_GetAPIKeys(t *testing.T) {
t.Run("Successfully returns all API keys", func(t *testing.T) {
user := portainer.User{ID: 1}
_, _, err := service.GenerateApiKey(user, "test-1")
require.NoError(t, err)
is.NoError(err)
_, _, err = service.GenerateApiKey(user, "test-2")
require.NoError(t, err)
is.NoError(err)
keys, err := service.GetAPIKeys(user.ID)
require.NoError(t, err)
is.NoError(err)
is.Len(keys, 2)
})
}
@@ -123,10 +122,10 @@ func Test_GetDigestUserAndKey(t *testing.T) {
t.Run("Successfully returns user and api key associated to digest", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-1")
require.NoError(t, err)
is.NoError(err)
userGot, apiKeyGot, err := service.GetDigestUserAndKey(apiKey.Digest)
require.NoError(t, err)
is.NoError(err)
is.Equal(user, userGot)
is.Equal(*apiKey, apiKeyGot)
})
@@ -134,10 +133,10 @@ func Test_GetDigestUserAndKey(t *testing.T) {
t.Run("Successfully caches user and api key associated to digest", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-1")
require.NoError(t, err)
is.NoError(err)
userGot, apiKeyGot, err := service.GetDigestUserAndKey(apiKey.Digest)
require.NoError(t, err)
is.NoError(err)
is.Equal(user, userGot)
is.Equal(*apiKey, apiKeyGot)
@@ -159,14 +158,14 @@ func Test_UpdateAPIKey(t *testing.T) {
user := portainer.User{ID: 1}
store.User().Create(&user)
_, apiKey, err := service.GenerateApiKey(user, "test-x")
require.NoError(t, err)
is.NoError(err)
apiKey.LastUsed = time.Now().UTC().Unix()
err = service.UpdateAPIKey(apiKey)
require.NoError(t, err)
is.NoError(err)
_, apiKeyGot, err := service.GetDigestUserAndKey(apiKey.Digest)
require.NoError(t, err)
is.NoError(err)
log.Debug().Str("wanted", fmt.Sprintf("%+v", apiKey)).Str("got", fmt.Sprintf("%+v", apiKeyGot)).Msg("")
@@ -175,7 +174,7 @@ func Test_UpdateAPIKey(t *testing.T) {
t.Run("Successfully updates api-key in cache upon api-key update", func(t *testing.T) {
_, apiKey, err := service.GenerateApiKey(portainer.User{ID: 1}, "test-x2")
require.NoError(t, err)
is.NoError(err)
_, apiKeyFromCache, ok := service.cache.Get(apiKey.Digest)
is.True(ok)
@@ -185,7 +184,7 @@ func Test_UpdateAPIKey(t *testing.T) {
is.NotEqual(*apiKey, apiKeyFromCache)
err = service.UpdateAPIKey(apiKey)
require.NoError(t, err)
is.NoError(err)
_, updatedAPIKeyFromCache, ok := service.cache.Get(apiKey.Digest)
is.True(ok)
@@ -203,30 +202,30 @@ func Test_DeleteAPIKey(t *testing.T) {
t.Run("Successfully updates the api-key", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-1")
require.NoError(t, err)
is.NoError(err)
_, apiKeyGot, err := service.GetDigestUserAndKey(apiKey.Digest)
require.NoError(t, err)
is.NoError(err)
is.Equal(*apiKey, apiKeyGot)
err = service.DeleteAPIKey(apiKey.ID)
require.NoError(t, err)
is.NoError(err)
_, _, err = service.GetDigestUserAndKey(apiKey.Digest)
require.Error(t, err)
is.Error(err)
})
t.Run("Successfully removes api-key from cache upon deletion", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-1")
require.NoError(t, err)
is.NoError(err)
_, apiKeyFromCache, ok := service.cache.Get(apiKey.Digest)
is.True(ok)
is.Equal(*apiKey, apiKeyFromCache)
err = service.DeleteAPIKey(apiKey.ID)
require.NoError(t, err)
is.NoError(err)
_, _, ok = service.cache.Get(apiKey.Digest)
is.False(ok)
@@ -244,10 +243,10 @@ func Test_InvalidateUserKeyCache(t *testing.T) {
// generate api keys
user := portainer.User{ID: 1}
_, apiKey1, err := service.GenerateApiKey(user, "test-1")
require.NoError(t, err)
is.NoError(err)
_, apiKey2, err := service.GenerateApiKey(user, "test-2")
require.NoError(t, err)
is.NoError(err)
// verify api keys are present in cache
_, apiKeyFromCache, ok := service.cache.Get(apiKey1.Digest)
@@ -274,11 +273,11 @@ func Test_InvalidateUserKeyCache(t *testing.T) {
// generate keys for 2 users
user1 := portainer.User{ID: 1}
_, apiKey1, err := service.GenerateApiKey(user1, "test-1")
require.NoError(t, err)
is.NoError(err)
user2 := portainer.User{ID: 2}
_, apiKey2, err := service.GenerateApiKey(user2, "test-2")
require.NoError(t, err)
is.NoError(err)
// verify keys in cache
_, apiKeyFromCache, ok := service.cache.Get(apiKey1.Digest)
+10 -31
View File
@@ -8,19 +8,15 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func listFiles(dir string) []string {
items := make([]string, 0)
filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if path == dir {
return nil
}
items = append(items, path)
return nil
})
@@ -30,21 +26,13 @@ func listFiles(dir string) []string {
func Test_shouldCreateArchive(t *testing.T) {
tmpdir := t.TempDir()
content := []byte("content")
err := os.WriteFile(path.Join(tmpdir, "outer"), content, 0600)
require.NoError(t, err)
os.WriteFile(path.Join(tmpdir, "outer"), content, 0600)
os.MkdirAll(path.Join(tmpdir, "dir"), 0700)
require.NoError(t, err)
err = os.WriteFile(path.Join(tmpdir, "dir", ".dotfile"), content, 0600)
require.NoError(t, err)
err = os.WriteFile(path.Join(tmpdir, "dir", "inner"), content, 0600)
require.NoError(t, err)
os.WriteFile(path.Join(tmpdir, "dir", ".dotfile"), content, 0600)
os.WriteFile(path.Join(tmpdir, "dir", "inner"), content, 0600)
gzPath, err := TarGzDir(tmpdir)
require.NoError(t, err)
assert.Nil(t, err)
assert.Equal(t, filepath.Join(tmpdir, filepath.Base(tmpdir)+".tar.gz"), gzPath)
extractionDir := t.TempDir()
@@ -57,8 +45,7 @@ func Test_shouldCreateArchive(t *testing.T) {
wasExtracted := func(p string) {
fullpath := path.Join(extractionDir, p)
assert.Contains(t, extractedFiles, fullpath)
copyContent, err := os.ReadFile(fullpath)
require.NoError(t, err)
copyContent, _ := os.ReadFile(fullpath)
assert.Equal(t, content, copyContent)
}
@@ -70,21 +57,13 @@ func Test_shouldCreateArchive(t *testing.T) {
func Test_shouldCreateArchive2(t *testing.T) {
tmpdir := t.TempDir()
content := []byte("content")
err := os.WriteFile(path.Join(tmpdir, "outer"), content, 0600)
require.NoError(t, err)
err = os.MkdirAll(path.Join(tmpdir, "dir"), 0700)
require.NoError(t, err)
err = os.WriteFile(path.Join(tmpdir, "dir", ".dotfile"), content, 0600)
require.NoError(t, err)
err = os.WriteFile(path.Join(tmpdir, "dir", "inner"), content, 0600)
require.NoError(t, err)
os.WriteFile(path.Join(tmpdir, "outer"), content, 0600)
os.MkdirAll(path.Join(tmpdir, "dir"), 0700)
os.WriteFile(path.Join(tmpdir, "dir", ".dotfile"), content, 0600)
os.WriteFile(path.Join(tmpdir, "dir", "inner"), content, 0600)
gzPath, err := TarGzDir(tmpdir)
require.NoError(t, err)
assert.Nil(t, err)
assert.Equal(t, filepath.Join(tmpdir, filepath.Base(tmpdir)+".tar.gz"), gzPath)
extractionDir := t.TempDir()
+1 -2
View File
@@ -5,7 +5,6 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestUnzipFile(t *testing.T) {
@@ -21,7 +20,7 @@ func TestUnzipFile(t *testing.T) {
err := UnzipFile("./testdata/sample_archive.zip", dir)
require.NoError(t, err)
assert.NoError(t, err)
archiveDir := dir + "/sample_archive"
assert.FileExists(t, filepath.Join(archiveDir, "0.txt"))
assert.FileExists(t, filepath.Join(archiveDir, "0", "1.txt"))
+1
View File
@@ -56,6 +56,7 @@ func CLIFlags() *portainer.CLIFlags {
PullLimitCheckDisabled: kingpin.Flag("pull-limit-check-disabled", "Pull limit check").Envar(portainer.PullLimitCheckDisabledEnvVar).Default(defaultPullLimitCheckDisabled).Bool(),
TrustedOrigins: kingpin.Flag("trusted-origins", "List of trusted origins for CSRF protection. Separate multiple origins with a comma.").Envar(portainer.TrustedOriginsEnvVar).String(),
CSP: kingpin.Flag("csp", "Content Security Policy (CSP) header").Envar(portainer.CSPEnvVar).Default("true").Bool(),
CompactDB: kingpin.Flag("compact-db", "Enable database compaction on startup").Envar(portainer.CompactDBEnvVar).Default("false").Bool(),
}
}
+1 -1
View File
@@ -84,7 +84,7 @@ func initFileService(dataStorePath string) portainer.FileService {
}
func initDataStore(flags *portainer.CLIFlags, secretKey []byte, fileService portainer.FileService, shutdownCtx context.Context) dataservices.DataStore {
connection, err := database.NewDatabase("boltdb", *flags.Data, secretKey)
connection, err := database.NewDatabase("boltdb", *flags.Data, secretKey, *flags.CompactDB)
if err != nil {
log.Fatal().Err(err).Msg("failed creating database connection")
}
+2 -3
View File
@@ -15,9 +15,8 @@ import (
"github.com/portainer/portainer/pkg/fips"
// Not allowed in FIPS mode
"golang.org/x/crypto/argon2" //nolint:depguard
"golang.org/x/crypto/scrypt" //nolint:depguard
"golang.org/x/crypto/argon2"
"golang.org/x/crypto/scrypt"
)
const (
+28 -42
View File
@@ -55,19 +55,17 @@ func Test_encryptAndDecrypt_withTheSamePassword(t *testing.T) {
encryptedFileWriter, _ := os.Create(encryptedFilePath)
err := encrypt(originFile, encryptedFileWriter, []byte(passphrase))
require.NoError(t, err, "Failed to encrypt a file")
require.Nil(t, err, "Failed to encrypt a file")
encryptedFileWriter.Close()
encryptedContent, err := os.ReadFile(encryptedFilePath)
require.NoError(t, err, "Couldn't read encrypted file")
require.Nil(t, err, "Couldn't read encrypted file")
assert.NotEqual(t, encryptedContent, content, "Content wasn't encrypted")
encryptedFileReader, err := os.Open(encryptedFilePath)
require.NoError(t, err)
encryptedFileReader, _ := os.Open(encryptedFilePath)
defer encryptedFileReader.Close()
decryptedFileWriter, err := os.Create(decryptedFilePath)
require.NoError(t, err)
decryptedFileWriter, _ := os.Create(decryptedFilePath)
defer decryptedFileWriter.Close()
decryptedReader, err := decrypt(encryptedFileReader, []byte(passphrase))
@@ -157,11 +155,11 @@ func Test_encryptAndDecrypt_withStrongPassphrase(t *testing.T) {
encryptedFileWriter, _ := os.Create(encryptedFilePath)
err := encrypt(originFile, encryptedFileWriter, []byte(passphrase))
require.NoError(t, err, "Failed to encrypt a file")
assert.Nil(t, err, "Failed to encrypt a file")
encryptedFileWriter.Close()
encryptedContent, err := os.ReadFile(encryptedFilePath)
require.NoError(t, err, "Couldn't read encrypted file")
assert.Nil(t, err, "Couldn't read encrypted file")
assert.NotEqual(t, encryptedContent, content, "Content wasn't encrypted")
encryptedFileReader, _ := os.Open(encryptedFilePath)
@@ -171,7 +169,7 @@ func Test_encryptAndDecrypt_withStrongPassphrase(t *testing.T) {
defer decryptedFileWriter.Close()
decryptedReader, err := decrypt(encryptedFileReader, []byte(passphrase))
require.NoError(t, err, "Failed to decrypt file")
assert.Nil(t, err, "Failed to decrypt file")
io.Copy(decryptedFileWriter, decryptedReader)
@@ -207,29 +205,25 @@ func Test_encryptAndDecrypt_withTheSamePasswordSmallFile(t *testing.T) {
encryptedFileWriter, _ := os.Create(encryptedFilePath)
err := encrypt(originFile, encryptedFileWriter, []byte("passphrase"))
require.NoError(t, err, "Failed to encrypt a file")
assert.Nil(t, err, "Failed to encrypt a file")
encryptedFileWriter.Close()
encryptedContent, err := os.ReadFile(encryptedFilePath)
require.NoError(t, err, "Couldn't read encrypted file")
assert.Nil(t, err, "Couldn't read encrypted file")
assert.NotEqual(t, encryptedContent, content, "Content wasn't encrypted")
encryptedFileReader, err := os.Open(encryptedFilePath)
require.NoError(t, err)
encryptedFileReader, _ := os.Open(encryptedFilePath)
defer encryptedFileReader.Close()
decryptedFileWriter, err := os.Create(decryptedFilePath)
require.NoError(t, err)
decryptedFileWriter, _ := os.Create(decryptedFilePath)
defer decryptedFileWriter.Close()
decryptedReader, err := decrypt(encryptedFileReader, []byte("passphrase"))
require.NoError(t, err, "Failed to decrypt file")
assert.Nil(t, err, "Failed to decrypt file")
_, err = io.Copy(decryptedFileWriter, decryptedReader)
require.NoError(t, err)
io.Copy(decryptedFileWriter, decryptedReader)
decryptedContent, err := os.ReadFile(decryptedFilePath)
require.NoError(t, err)
decryptedContent, _ := os.ReadFile(decryptedFilePath)
assert.Equal(t, content, decryptedContent, "Original and decrypted content should match")
}
@@ -253,40 +247,32 @@ func Test_encryptAndDecrypt_withEmptyPassword(t *testing.T) {
)
content := randBytes(1024 * 50)
err := os.WriteFile(originFilePath, content, 0600)
require.NoError(t, err)
os.WriteFile(originFilePath, content, 0600)
originFile, err := os.Open(originFilePath)
require.NoError(t, err)
originFile, _ := os.Open(originFilePath)
defer originFile.Close()
encryptedFileWriter, err := os.Create(encryptedFilePath)
require.NoError(t, err)
encryptedFileWriter, _ := os.Create(encryptedFilePath)
defer encryptedFileWriter.Close()
err = encrypt(originFile, encryptedFileWriter, []byte(""))
require.NoError(t, err, "Failed to encrypt a file")
err := encrypt(originFile, encryptedFileWriter, []byte(""))
assert.Nil(t, err, "Failed to encrypt a file")
encryptedContent, err := os.ReadFile(encryptedFilePath)
require.NoError(t, err, "Couldn't read encrypted file")
assert.Nil(t, err, "Couldn't read encrypted file")
assert.NotEqual(t, encryptedContent, content, "Content wasn't encrypted")
encryptedFileReader, err := os.Open(encryptedFilePath)
require.NoError(t, err)
encryptedFileReader, _ := os.Open(encryptedFilePath)
defer encryptedFileReader.Close()
decryptedFileWriter, err := os.Create(decryptedFilePath)
require.NoError(t, err)
decryptedFileWriter, _ := os.Create(decryptedFilePath)
defer decryptedFileWriter.Close()
decryptedReader, err := decrypt(encryptedFileReader, []byte(""))
require.NoError(t, err, "Failed to decrypt file")
assert.Nil(t, err, "Failed to decrypt file")
_, err = io.Copy(decryptedFileWriter, decryptedReader)
require.NoError(t, err)
io.Copy(decryptedFileWriter, decryptedReader)
decryptedContent, err := os.ReadFile(decryptedFilePath)
require.NoError(t, err)
decryptedContent, _ := os.ReadFile(decryptedFilePath)
assert.Equal(t, content, decryptedContent, "Original and decrypted content should match")
}
@@ -319,9 +305,9 @@ func Test_decryptWithDifferentPassphrase_shouldProduceWrongResult(t *testing.T)
defer encryptedFileWriter.Close()
err := encrypt(originFile, encryptedFileWriter, []byte("passphrase"))
require.NoError(t, err, "Failed to encrypt a file")
assert.Nil(t, err, "Failed to encrypt a file")
encryptedContent, err := os.ReadFile(encryptedFilePath)
require.NoError(t, err, "Couldn't read encrypted file")
assert.Nil(t, err, "Couldn't read encrypted file")
assert.NotEqual(t, encryptedContent, content, "Content wasn't encrypted")
encryptedFileReader, _ := os.Open(encryptedFilePath)
@@ -331,7 +317,7 @@ func Test_decryptWithDifferentPassphrase_shouldProduceWrongResult(t *testing.T)
defer decryptedFileWriter.Close()
_, err = decrypt(encryptedFileReader, []byte("garbage"))
require.Error(t, err, "Should not allow decrypt with wrong passphrase")
assert.NotNil(t, err, "Should not allow decrypt with wrong passphrase")
}
t.Run("fips", func(t *testing.T) {
+3 -3
View File
@@ -11,12 +11,12 @@ func TestCreateSignature(t *testing.T) {
privKey, pubKey, err := s.GenerateKeyPair()
require.NoError(t, err)
require.NotEmpty(t, privKey)
require.NotEmpty(t, pubKey)
require.Greater(t, len(privKey), 0)
require.Greater(t, len(pubKey), 0)
m := "test message"
r, err := s.CreateSignature(m)
require.NoError(t, err)
require.NotEqual(t, r, m)
require.NotEmpty(t, r)
require.Greater(t, len(r), 0)
}
+1 -2
View File
@@ -1,8 +1,7 @@
package crypto
import (
// Not allowed in FIPS mode
"golang.org/x/crypto/bcrypt" //nolint:depguard
"golang.org/x/crypto/bcrypt"
)
// Service represents a service for encrypting/hashing data.
+2 -2
View File
@@ -44,7 +44,7 @@ func TestCreateTLSConfigurationFIPS(t *testing.T) {
func TestCreateTLSConfigurationFromBytes(t *testing.T) {
// No TLS
config, err := CreateTLSConfigurationFromBytes(false, nil, nil, nil, false, false)
require.NoError(t, err)
require.Nil(t, err)
require.Nil(t, config)
// Skip TLS client/server verifications
@@ -61,7 +61,7 @@ func TestCreateTLSConfigurationFromBytes(t *testing.T) {
func TestCreateTLSConfigurationFromDisk(t *testing.T) {
// No TLS
config, err := CreateTLSConfigurationFromDisk(portainer.TLSConfiguration{})
require.NoError(t, err)
require.Nil(t, err)
require.Nil(t, config)
// Skip TLS verifications
+68 -8
View File
@@ -21,6 +21,9 @@ import (
const (
DatabaseFileName = "portainer.db"
EncryptedDatabaseFileName = "portainer.edb"
txMaxSize = 65536
compactedSuffix = ".compacted"
)
var (
@@ -35,6 +38,7 @@ type DbConnection struct {
InitialMmapSize int
EncryptionKey []byte
isEncrypted bool
Compact bool
*bolt.DB
}
@@ -132,15 +136,8 @@ func (connection *DbConnection) NeedsEncryptionMigration() (bool, error) {
func (connection *DbConnection) Open() error {
log.Info().Str("filename", connection.GetDatabaseFileName()).Msg("loading PortainerDB")
// Now we open the db
databasePath := connection.GetDatabaseFilePath()
db, err := bolt.Open(databasePath, 0600, &bolt.Options{
Timeout: 1 * time.Second,
InitialMmapSize: connection.InitialMmapSize,
FreelistType: bolt.FreelistMapType,
NoFreelistSync: true,
})
db, err := bolt.Open(databasePath, 0600, connection.boltOptions(connection.Compact))
if err != nil {
return err
}
@@ -149,6 +146,24 @@ func (connection *DbConnection) Open() error {
db.MaxBatchDelay = connection.MaxBatchDelay
connection.DB = db
if connection.Compact {
log.Info().Msg("compacting database")
if err := connection.compact(); err != nil {
log.Error().Err(err).Msg("failed to compact database")
// Close the read-only database and re-open in read-write mode
if err := connection.Close(); err != nil {
log.Warn().Err(err).Msg("failure to close the database after failed compaction")
}
connection.Compact = false
return connection.Open()
} else {
log.Info().Msg("database compaction completed")
}
}
return nil
}
@@ -414,3 +429,48 @@ func (connection *DbConnection) RestoreMetadata(s map[string]any) error {
return err
}
// compact attempts to compact the database and replace it iff it succeeds
func (connection *DbConnection) compact() (err error) {
compactedPath := connection.GetDatabaseFilePath() + compactedSuffix
if err := os.Remove(compactedPath); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("failure to remove an existing compacted database: %w", err)
}
compactedDB, err := bolt.Open(compactedPath, 0o600, connection.boltOptions(false))
if err != nil {
return fmt.Errorf("failure to create the compacted database: %w", err)
}
compactedDB.MaxBatchSize = connection.MaxBatchSize
compactedDB.MaxBatchDelay = connection.MaxBatchDelay
if err := bolt.Compact(compactedDB, connection.DB, txMaxSize); err != nil {
return fmt.Errorf("failure to compact the database: %w",
errors.Join(err, compactedDB.Close(), os.Remove(compactedPath)))
}
if err := os.Rename(compactedPath, connection.GetDatabaseFilePath()); err != nil {
return fmt.Errorf("failure to move the compacted database: %w",
errors.Join(err, compactedDB.Close(), os.Remove(compactedPath)))
}
if err := connection.Close(); err != nil {
log.Warn().Err(err).Msg("failure to close the database after compaction")
}
connection.DB = compactedDB
return nil
}
func (connection *DbConnection) boltOptions(readOnly bool) *bolt.Options {
return &bolt.Options{
Timeout: 1 * time.Second,
InitialMmapSize: connection.InitialMmapSize,
FreelistType: bolt.FreelistMapType,
NoFreelistSync: true,
ReadOnly: readOnly,
}
}
+60
View File
@@ -5,7 +5,11 @@ import (
"path"
"testing"
"github.com/portainer/portainer/api/filesystem"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.etcd.io/bbolt"
)
func Test_NeedsEncryptionMigration(t *testing.T) {
@@ -119,3 +123,59 @@ func Test_NeedsEncryptionMigration(t *testing.T) {
})
}
}
func TestDBCompaction(t *testing.T) {
db := &DbConnection{Path: t.TempDir()}
err := db.Open()
require.NoError(t, err)
err = db.Update(func(tx *bbolt.Tx) error {
b, err := tx.CreateBucketIfNotExists([]byte("testbucket"))
if err != nil {
return err
}
b.Put([]byte("key"), []byte("value"))
return nil
})
require.NoError(t, err)
err = db.Close()
require.NoError(t, err)
// Reopen the DB to trigger compaction
db.Compact = true
err = db.Open()
require.NoError(t, err)
// Check that the data is still there
err = db.View(func(tx *bbolt.Tx) error {
b := tx.Bucket([]byte("testbucket"))
if b == nil {
return nil
}
val := b.Get([]byte("key"))
require.Equal(t, []byte("value"), val)
return nil
})
require.NoError(t, err)
err = db.Close()
require.NoError(t, err)
// Failures
compactedPath := db.GetDatabaseFilePath() + compactedSuffix
err = os.Mkdir(compactedPath, 0o755)
require.NoError(t, err)
f, err := os.Create(filesystem.JoinPaths(compactedPath, "somefile"))
require.NoError(t, err)
require.NoError(t, f.Close())
err = db.Open()
require.NoError(t, err)
}
+4 -4
View File
@@ -94,7 +94,7 @@ func Test_MarshalObjectUnencrypted(t *testing.T) {
for _, test := range tests {
t.Run(fmt.Sprintf("%s -> %s", test.object, test.expected), func(t *testing.T) {
data, err := conn.MarshalObject(test.object)
require.NoError(t, err)
is.NoError(err)
is.Equal(test.expected, string(data))
})
}
@@ -135,7 +135,7 @@ func Test_UnMarshalObjectUnencrypted(t *testing.T) {
t.Run(fmt.Sprintf("%s -> %s", test.object, test.expected), func(t *testing.T) {
var object string
err := conn.UnmarshalObject(test.object, &object)
require.NoError(t, err)
is.NoError(err)
is.Equal(test.expected, object)
})
}
@@ -172,12 +172,12 @@ func Test_ObjectMarshallingEncrypted(t *testing.T) {
t.Run(fmt.Sprintf("%s -> %s", test.object, test.expected), func(t *testing.T) {
data, err := conn.MarshalObject(test.object)
require.NoError(t, err)
is.NoError(err)
var object []byte
err = conn.UnmarshalObject(data, &object)
require.NoError(t, err)
is.NoError(err)
is.Equal(test.object, object)
})
}
+2 -1
View File
@@ -8,11 +8,12 @@ import (
)
// NewDatabase should use config options to return a connection to the requested database
func NewDatabase(storeType, storePath string, encryptionKey []byte) (connection portainer.Connection, err error) {
func NewDatabase(storeType, storePath string, encryptionKey []byte, compact bool) (connection portainer.Connection, err error) {
if storeType == "boltdb" {
return &boltdb.DbConnection{
Path: storePath,
EncryptionKey: encryptionKey,
Compact: compact,
}, nil
}
@@ -136,5 +136,5 @@ func TestEndpointRelations(t *testing.T) {
require.NoError(t, service.Create(&portainer.EndpointRelation{EndpointID: 1}))
rels, err := service.EndpointRelations()
require.NoError(t, err)
require.Len(t, rels, 1)
require.Equal(t, 1, len(rels))
}
+6 -7
View File
@@ -4,18 +4,17 @@ import (
"testing"
"time"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/datastore"
"github.com/portainer/portainer/api/filesystem"
"github.com/gofrs/uuid"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/filesystem"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func newGuidString(t *testing.T) string {
uuid, err := uuid.NewV4()
require.NoError(t, err)
assert.NoError(t, err)
return uuid.String()
}
@@ -42,7 +41,7 @@ func TestService_StackByWebhookID(t *testing.T) {
// can find a stack by webhook ID
got, err := store.StackService.StackByWebhookID(webhookID)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, stack, *got)
// returns nil and object not found error if there's no stack associated with the webhook
@@ -95,10 +94,10 @@ func Test_RefreshableStacks(t *testing.T) {
for _, stack := range []*portainer.Stack{&staticStack, &stackWithWebhook, &refreshableStack} {
err := store.Stack().Create(stack)
require.NoError(t, err)
assert.NoError(t, err)
}
stacks, err := store.Stack().RefreshableStacks()
require.NoError(t, err)
assert.NoError(t, err)
assert.ElementsMatch(t, []portainer.Stack{refreshableStack}, stacks)
}
+3 -5
View File
@@ -5,9 +5,7 @@ import (
"github.com/portainer/portainer/api/dataservices/errors"
"github.com/portainer/portainer/api/datastore"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_teamByName(t *testing.T) {
@@ -15,7 +13,7 @@ func Test_teamByName(t *testing.T) {
_, store := datastore.MustNewTestStore(t, true, true)
_, err := store.Team().TeamByName("name")
require.ErrorIs(t, err, errors.ErrObjectNotFound)
assert.ErrorIs(t, err, errors.ErrObjectNotFound)
})
@@ -31,7 +29,7 @@ func Test_teamByName(t *testing.T) {
teamBuilder.createNew("name1")
_, err := store.Team().TeamByName("name")
require.ErrorIs(t, err, errors.ErrObjectNotFound)
assert.ErrorIs(t, err, errors.ErrObjectNotFound)
})
t.Run("When there is an object with the same name should return the object", func(t *testing.T) {
@@ -46,7 +44,7 @@ func Test_teamByName(t *testing.T) {
expectedTeam := teamBuilder.createNew("name1")
team, err := store.Team().TeamByName("name1")
require.NoError(t, err, "TeamByName should succeed")
assert.NoError(t, err, "TeamByName should succeed")
assert.Equal(t, expectedTeam, team)
})
}
+3 -2
View File
@@ -5,14 +5,15 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/database/models"
"github.com/stretchr/testify/require"
"github.com/rs/zerolog/log"
)
func TestStoreCreation(t *testing.T) {
_, store := MustNewTestStore(t, true, true)
require.NotNil(t, store)
if store == nil {
t.Fatal("Expect to create a store")
}
v, err := store.VersionService.Version()
if err != nil {
+74 -42
View File
@@ -6,14 +6,12 @@ import (
"strings"
"testing"
"github.com/dchest/uniuri"
"github.com/pkg/errors"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/chisel"
"github.com/portainer/portainer/api/crypto"
"github.com/dchest/uniuri"
"github.com/pkg/errors"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const (
@@ -32,21 +30,45 @@ func TestStoreFull(t *testing.T) {
_, store := MustNewTestStore(t, true, true)
testCases := map[string]func(t *testing.T){
"User Accounts": store.testUserAccounts,
"Environments": store.testEnvironments,
"Settings": store.testSettings,
"SSL Settings": store.testSSLSettings,
"Tunnel Server": store.testTunnelServer,
"Custom Templates": store.testCustomTemplates,
"Registries": store.testRegistries,
"Resource Control": store.testResourceControl,
"Schedules": store.testSchedules,
"Tags": store.testTags,
"User Accounts": func(t *testing.T) {
store.testUserAccounts(t)
},
"Environments": func(t *testing.T) {
store.testEnvironments(t)
},
"Settings": func(t *testing.T) {
store.testSettings(t)
},
"SSL Settings": func(t *testing.T) {
store.testSSLSettings(t)
},
"Tunnel Server": func(t *testing.T) {
store.testTunnelServer(t)
},
"Custom Templates": func(t *testing.T) {
store.testCustomTemplates(t)
},
"Registries": func(t *testing.T) {
store.testRegistries(t)
},
"Resource Control": func(t *testing.T) {
store.testResourceControl(t)
},
"Schedules": func(t *testing.T) {
store.testSchedules(t)
},
"Tags": func(t *testing.T) {
store.testTags(t)
},
// "Test Title": func(t *testing.T) {
// },
}
for name, test := range testCases {
t.Run(name, test)
}
}
func (store *Store) testEnvironments(t *testing.T) {
@@ -145,7 +167,7 @@ func (store *Store) CreateEndpoint(t *testing.T, name string, endpointType porta
store.Endpoint().Create(expectedEndpoint)
endpoint, err := store.Endpoint().Endpoint(id)
require.NoError(t, err, "Endpoint() should not return an error")
is.NoError(err, "Endpoint() should not return an error")
is.Equal(expectedEndpoint, endpoint, "endpoint should be the same")
return endpoint.ID
@@ -172,7 +194,7 @@ func (store *Store) testSSLSettings(t *testing.T) {
store.SSLSettings().UpdateSettings(ssl)
settings, err := store.SSLSettings().Settings()
require.NoError(t, err, "Get sslsettings should succeed")
is.NoError(err, "Get sslsettings should succeed")
is.Equal(ssl, settings, "Stored SSLSettings should be the same as what is read out")
}
@@ -181,27 +203,27 @@ func (store *Store) testTunnelServer(t *testing.T) {
expectPrivateKeySeed := uniuri.NewLen(16)
err := store.TunnelServer().UpdateInfo(&portainer.TunnelServerInfo{PrivateKeySeed: expectPrivateKeySeed})
require.NoError(t, err, "UpdateInfo should have succeeded")
is.NoError(err, "UpdateInfo should have succeeded")
serverInfo, err := store.TunnelServer().Info()
require.NoError(t, err, "Info should have succeeded")
is.NoError(err, "Info should have succeeded")
is.Equal(expectPrivateKeySeed, serverInfo.PrivateKeySeed, "hashed passwords should not differ")
}
// add users, read them back and check the details are unchanged
func (store *Store) testUserAccounts(t *testing.T) {
err := store.createAccount(adminUsername, adminPassword, portainer.AdministratorRole)
require.NoError(t, err, "CreateAccount should succeed")
is := assert.New(t)
err = store.checkAccount(adminUsername, adminPassword, portainer.AdministratorRole)
require.NoError(t, err, "Account failure")
err := store.createAccount(adminUsername, adminPassword, portainer.AdministratorRole)
is.NoError(err, "CreateAccount should succeed")
store.checkAccount(adminUsername, adminPassword, portainer.AdministratorRole)
is.NoError(err, "Account failure")
err = store.createAccount(standardUsername, standardPassword, portainer.StandardUserRole)
require.NoError(t, err, "CreateAccount should succeed")
err = store.checkAccount(standardUsername, standardPassword, portainer.StandardUserRole)
require.NoError(t, err, "Account failure")
is.NoError(err, "CreateAccount should succeed")
store.checkAccount(standardUsername, standardPassword, portainer.StandardUserRole)
is.NoError(err, "Account failure")
}
// create an account with the provided details
@@ -216,7 +238,12 @@ func (store *Store) createAccount(username, password string, role portainer.User
return err
}
return store.User().Create(user)
err = store.User().Create(user)
if err != nil {
return err
}
return nil
}
func (store *Store) checkAccount(username, expectPassword string, expectRole portainer.UserRole) error {
@@ -233,7 +260,12 @@ func (store *Store) checkAccount(username, expectPassword string, expectRole por
// Check the password
cs := crypto.Service{}
if cs.CompareHashAndData(user.Password, expectPassword) != nil {
expectPasswordHash, err := cs.Hash(expectPassword)
if err != nil {
return errors.Wrap(err, "hash failed")
}
if user.Password != expectPasswordHash {
return fmt.Errorf("%s user password hash failure", user.Username)
}
@@ -245,7 +277,7 @@ func (store *Store) testSettings(t *testing.T) {
// since many settings are default and basically nil, I'm going to update some and read them back
expectedSettings, err := store.Settings().Settings()
require.NoError(t, err, "Settings() should not return an error")
is.NoError(err, "Settings() should not return an error")
expectedSettings.TemplatesURL = "http://portainer.io/application-templates"
expectedSettings.HelmRepositoryURL = "http://portainer.io/helm-repository"
expectedSettings.EdgeAgentCheckinInterval = 60
@@ -259,10 +291,10 @@ func (store *Store) testSettings(t *testing.T) {
expectedSettings.SnapshotInterval = "10m"
err = store.Settings().UpdateSettings(expectedSettings)
require.NoError(t, err, "UpdateSettings() should succeed")
is.NoError(err, "UpdateSettings() should succeed")
settings, err := store.Settings().Settings()
require.NoError(t, err, "Settings() should not return an error")
is.NoError(err, "Settings() should not return an error")
is.Equal(expectedSettings, settings, "stored settings should match")
}
@@ -285,7 +317,7 @@ func (store *Store) testCustomTemplates(t *testing.T) {
customTemplate.Create(expectedTemplate)
actualTemplate, err := customTemplate.Read(expectedTemplate.ID)
require.NoError(t, err, "CustomTemplate should not return an error")
is.NoError(err, "CustomTemplate should not return an error")
is.Equal(expectedTemplate, actualTemplate, "expected and actual template do not match")
}
@@ -313,17 +345,17 @@ func (store *Store) testRegistries(t *testing.T) {
}
err := regService.Create(reg1)
require.NoError(t, err)
is.NoError(err)
err = regService.Create(reg2)
require.NoError(t, err)
is.NoError(err)
actualReg1, err := regService.Read(reg1.ID)
require.NoError(t, err)
is.NoError(err)
is.Equal(reg1, actualReg1, "registries differ")
actualReg2, err := regService.Read(reg2.ID)
require.NoError(t, err)
is.NoError(err)
is.Equal(reg2, actualReg2, "registries differ")
}
@@ -346,10 +378,10 @@ func (store *Store) testSchedules(t *testing.T) {
}
err := schedule.CreateSchedule(s)
require.NoError(t, err, "CreateSchedule should succeed")
is.NoError(err, "CreateSchedule should succeed")
actual, err := schedule.Schedule(s.ID)
require.NoError(t, err, "schedule should be found")
is.NoError(err, "schedule should be found")
is.Equal(s, actual, "schedules differ")
}
@@ -369,16 +401,16 @@ func (store *Store) testTags(t *testing.T) {
}
err := tags.Create(tag1)
require.NoError(t, err, "Tags.Create should succeed")
is.NoError(err, "Tags.Create should succeed")
err = tags.Create(tag2)
require.NoError(t, err, "Tags.Create should succeed")
is.NoError(err, "Tags.Create should succeed")
actual, err := tags.Read(tag1.ID)
require.NoError(t, err, "tag1 should be found")
is.NoError(err, "tag1 should be found")
is.Equal(tag1, actual, "tags differ")
actual, err = tags.Read(tag2.ID)
require.NoError(t, err, "tag2 should be found")
is.NoError(err, "tag2 should be found")
is.Equal(tag2, actual, "tags differ")
}
+7 -9
View File
@@ -6,9 +6,7 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/datastore/migrator"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestMigrateStackEntryPoint(t *testing.T) {
@@ -30,25 +28,25 @@ func TestMigrateStackEntryPoint(t *testing.T) {
for _, s := range stacks {
err := stackService.Create(s)
require.NoError(t, err, "failed to create stack")
assert.NoError(t, err, "failed to create stack")
}
s, err := stackService.Read(1)
require.NoError(t, err)
assert.NoError(t, err)
assert.Nil(t, s.GitConfig, "first stack should not have git config")
s, err = stackService.Read(2)
require.NoError(t, err)
assert.Empty(t, s.GitConfig.ConfigFilePath, "not migrated yet migrated")
assert.NoError(t, err)
assert.Equal(t, "", s.GitConfig.ConfigFilePath, "not migrated yet migrated")
err = migrator.MigrateStackEntryPoint(stackService)
require.NoError(t, err, "failed to migrate entry point to Git ConfigFilePath")
assert.NoError(t, err, "failed to migrate entry point to Git ConfigFilePath")
s, err = stackService.Read(1)
require.NoError(t, err)
assert.NoError(t, err)
assert.Nil(t, s.GitConfig, "first stack should not have git config")
s, err = stackService.Read(2)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, "dir/sub/compose.yml", s.GitConfig.ConfigFilePath, "second stack should have config file path migrated")
}
+2 -2
View File
@@ -39,7 +39,7 @@ func TestMigrateEdgeGroupEndpointsToRoars_2_33_0Idempotency(t *testing.T) {
migratedEdgeGroup, err := edgeGroupService.Read(edgeGroup.ID)
require.NoError(t, err)
require.Empty(t, migratedEdgeGroup.Endpoints)
require.Len(t, migratedEdgeGroup.Endpoints, 0)
require.Equal(t, len(edgeGroup.Endpoints), migratedEdgeGroup.EndpointIDs.Len())
// Run migration again to ensure the results didn't change
@@ -50,6 +50,6 @@ func TestMigrateEdgeGroupEndpointsToRoars_2_33_0Idempotency(t *testing.T) {
migratedEdgeGroup, err = edgeGroupService.Read(edgeGroup.ID)
require.NoError(t, err)
require.Empty(t, migratedEdgeGroup.Endpoints)
require.Len(t, migratedEdgeGroup.Endpoints, 0)
require.Equal(t, len(edgeGroup.Endpoints), migratedEdgeGroup.EndpointIDs.Len())
}
-2
View File
@@ -258,8 +258,6 @@ func (m *Migrator) initMigrations() {
m.addMigrations("2.33.1", m.migrateEdgeGroupEndpointsToRoars_2_33_0)
// WARNING: do not change migrations that have already been released!
// Add new migrations above...
// One function per migration, each versions migration funcs in the same file.
}
@@ -22,9 +22,8 @@ func TestMigrateGPUs(t *testing.T) {
if strings.HasSuffix(r.URL.Path, "/containers/json") {
containerSummary := []container.Summary{{ID: "container1"}}
if err := json.NewEncoder(w).Encode(containerSummary); err != nil {
w.WriteHeader(http.StatusInternalServerError)
}
err := json.NewEncoder(w).Encode(containerSummary)
require.NoError(t, err)
return
}
@@ -42,9 +41,8 @@ func TestMigrateGPUs(t *testing.T) {
},
}
if err := json.NewEncoder(w).Encode(container); err != nil {
w.WriteHeader(http.StatusInternalServerError)
}
err := json.NewEncoder(w).Encode(container)
require.NoError(t, err)
}))
defer srv.Close()
@@ -131,12 +129,12 @@ func TestPostInitMigrate_PendingActionsCreated(t *testing.T) {
EdgeID: "edgeID",
}
err := store.Endpoint().Create(endpoint)
require.NoError(t, err, "error creating endpoint")
is.NoError(err, "error creating endpoint")
// Create any existing pending actions
for _, action := range tt.existingPendingActions {
err = store.PendingActions().Create(action)
require.NoError(t, err, "error creating pending action")
is.NoError(err, "error creating pending action")
}
migrator := NewPostInitMigrator(
@@ -148,11 +146,11 @@ func TestPostInitMigrate_PendingActionsCreated(t *testing.T) {
)
err = migrator.PostInitMigrate()
require.NoError(t, err, "PostInitMigrate should not return error")
is.NoError(err, "PostInitMigrate should not return error")
// Verify the results
pendingActions, err := store.PendingActions().ReadAll()
require.NoError(t, err, "error reading pending actions")
is.NoError(err, "error reading pending actions")
is.Len(pendingActions, tt.expectedPendingActions, "unexpected number of pending actions")
// If we expect any actions, verify at least one has the expected action type
@@ -162,11 +160,9 @@ func TestPostInitMigrate_PendingActionsCreated(t *testing.T) {
if action.Action == tt.expectedAction {
hasExpectedAction = true
is.Equal(endpoint.ID, action.EndpointID, "action should reference correct endpoint")
break
}
}
is.True(hasExpectedAction, "should have found action of expected type")
}
})
@@ -83,6 +83,7 @@
"MigrateIngresses": true
},
"PublicURL": "",
"QueryDate": 0,
"SecuritySettings": {
"allowBindMountsForRegularUsers": true,
"allowContainerCapabilitiesForRegularUsers": true,
@@ -614,7 +615,7 @@
"RequiredPasswordLength": 12
},
"KubeconfigExpiry": "0",
"KubectlShellImage": "portainer/kubectl-shell:2.34.0",
"KubectlShellImage": "portainer/kubectl-shell:2.33.3",
"LDAPSettings": {
"AnonymousMode": true,
"AutoCreateUsers": true,
@@ -943,7 +944,7 @@
}
],
"version": {
"VERSION": "{\"SchemaVersion\":\"2.34.0\",\"MigratorCount\":0,\"Edition\":1,\"InstanceID\":\"463d5c47-0ea5-4aca-85b1-405ceefee254\"}"
"VERSION": "{\"SchemaVersion\":\"2.33.3\",\"MigratorCount\":0,\"Edition\":1,\"InstanceID\":\"463d5c47-0ea5-4aca-85b1-405ceefee254\"}"
},
"webhooks": null
}
+1 -1
View File
@@ -44,7 +44,7 @@ func NewTestStore(t testing.TB, init, secure bool) (bool, *Store, func(), error)
secretKey = nil
}
connection, err := database.NewDatabase("boltdb", storePath, secretKey)
connection, err := database.NewDatabase("boltdb", storePath, secretKey, false)
if err != nil {
panic(err)
}
+37
View File
@@ -0,0 +1,37 @@
package docker
import "github.com/docker/docker/api/types"
type ContainerStats struct {
Running int `json:"running"`
Stopped int `json:"stopped"`
Healthy int `json:"healthy"`
Unhealthy int `json:"unhealthy"`
Total int `json:"total"`
}
func CalculateContainerStats(containers []types.Container) ContainerStats {
var running, stopped, healthy, unhealthy int
for _, container := range containers {
switch container.State {
case "running":
running++
case "healthy":
running++
healthy++
case "unhealthy":
running++
unhealthy++
case "exited", "stopped":
stopped++
}
}
return ContainerStats{
Running: running,
Stopped: stopped,
Healthy: healthy,
Unhealthy: unhealthy,
Total: len(containers),
}
}
+27
View File
@@ -0,0 +1,27 @@
package docker
import (
"testing"
"github.com/docker/docker/api/types"
"github.com/stretchr/testify/assert"
)
func TestCalculateContainerStats(t *testing.T) {
containers := []types.Container{
{State: "running"},
{State: "running"},
{State: "exited"},
{State: "stopped"},
{State: "healthy"},
{State: "unhealthy"},
}
stats := CalculateContainerStats(containers)
assert.Equal(t, 4, stats.Running)
assert.Equal(t, 2, stats.Stopped)
assert.Equal(t, 1, stats.Healthy)
assert.Equal(t, 1, stats.Unhealthy)
assert.Equal(t, 6, stats.Total)
}
+10 -14
View File
@@ -4,7 +4,6 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestImageParser(t *testing.T) {
@@ -15,7 +14,7 @@ func TestImageParser(t *testing.T) {
image, err := ParseImage(ParseImageOptions{
Name: "portainer/portainer-ee",
})
require.NoError(t, err)
is.NoError(err, "")
is.Equal("docker.io/portainer/portainer-ee:latest", image.FullName())
is.Equal("portainer/portainer-ee", image.Opts.Name)
is.Equal("latest", image.Tag)
@@ -31,10 +30,10 @@ func TestImageParser(t *testing.T) {
image, err := ParseImage(ParseImageOptions{
Name: "gcr.io/k8s-minikube/kicbase@sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b2",
})
require.NoError(t, err)
is.NoError(err, "")
is.Equal("gcr.io/k8s-minikube/kicbase@sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b2", image.FullName())
is.Equal("gcr.io/k8s-minikube/kicbase@sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b2", image.Opts.Name)
is.Empty(image.Tag)
is.Equal("", image.Tag)
is.Equal("k8s-minikube/kicbase", image.Path)
is.Equal("gcr.io", image.Domain)
is.Equal("https://gcr.io/k8s-minikube/kicbase", image.HubLink)
@@ -48,7 +47,7 @@ func TestImageParser(t *testing.T) {
image, err := ParseImage(ParseImageOptions{
Name: "gcr.io/k8s-minikube/kicbase:v0.0.30@sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b2",
})
require.NoError(t, err)
is.NoError(err, "")
is.Equal("gcr.io/k8s-minikube/kicbase:v0.0.30", image.FullName())
is.Equal("gcr.io/k8s-minikube/kicbase:v0.0.30@sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b2", image.Opts.Name)
is.Equal("v0.0.30", image.Tag)
@@ -69,9 +68,8 @@ func TestUpdateParsedImage(t *testing.T) {
image, err := ParseImage(ParseImageOptions{
Name: "gcr.io/k8s-minikube/kicbase:v0.0.30@sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b2",
})
require.NoError(t, err)
err = image.WithTag("v0.0.31")
require.NoError(t, err)
is.NoError(err, "")
_ = image.WithTag("v0.0.31")
is.Equal("gcr.io/k8s-minikube/kicbase:v0.0.31", image.FullName())
is.Equal("gcr.io/k8s-minikube/kicbase:v0.0.30@sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b2", image.Opts.Name)
is.Equal("v0.0.31", image.Tag)
@@ -88,9 +86,8 @@ func TestUpdateParsedImage(t *testing.T) {
image, err := ParseImage(ParseImageOptions{
Name: "gcr.io/k8s-minikube/kicbase:v0.0.30@sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b2",
})
require.NoError(t, err)
err = image.WithDigest("sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b3")
require.NoError(t, err)
is.NoError(err, "")
_ = image.WithDigest("sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b3")
is.Equal("gcr.io/k8s-minikube/kicbase:v0.0.30", image.FullName())
is.Equal("gcr.io/k8s-minikube/kicbase:v0.0.30@sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b2", image.Opts.Name)
is.Equal("v0.0.30", image.Tag)
@@ -107,9 +104,8 @@ func TestUpdateParsedImage(t *testing.T) {
image, err := ParseImage(ParseImageOptions{
Name: "gcr.io/k8s-minikube/kicbase:v0.0.30@sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b2",
})
require.NoError(t, err)
err = image.TrimDigest()
require.NoError(t, err)
is.NoError(err, "")
_ = image.TrimDigest()
is.Equal("gcr.io/k8s-minikube/kicbase:v0.0.30", image.FullName())
is.Equal("gcr.io/k8s-minikube/kicbase:v0.0.30@sha256:02c921df998f95e849058af14de7045efc3954d90320967418a0d1f182bbc0b2", image.Opts.Name)
is.Equal("v0.0.30", image.Tag)
+12 -14
View File
@@ -4,9 +4,7 @@ import (
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestFindBestMatchNeedAuthRegistry(t *testing.T) {
@@ -17,9 +15,9 @@ func TestFindBestMatchNeedAuthRegistry(t *testing.T) {
registries := []portainer.Registry{createNewRegistry("docker.io", "USERNAME", false),
createNewRegistry("hub-mirror.c.163.com", "", false)}
r, err := findBestMatchRegistry(image, registries)
require.NoError(t, err)
is.NotNil(r)
is.False(r.Authentication)
is.NoError(err, "")
is.NotNil(r, "")
is.False(r.Authentication, "")
is.Equal("docker.io", r.URL)
})
@@ -28,9 +26,9 @@ func TestFindBestMatchNeedAuthRegistry(t *testing.T) {
registries := []portainer.Registry{createNewRegistry("docker.io", "", false),
createNewRegistry("hub-mirror.c.163.com", "USERNAME", false)}
r, err := findBestMatchRegistry(image, registries)
require.NoError(t, err)
is.NotNil(r)
is.False(r.Authentication)
is.NoError(err, "")
is.NotNil(r, "")
is.False(r.Authentication, "")
is.Equal("docker.io", r.URL)
})
@@ -39,9 +37,9 @@ func TestFindBestMatchNeedAuthRegistry(t *testing.T) {
registries := []portainer.Registry{createNewRegistry("docker.io", "USERNAME", true),
createNewRegistry("hub-mirror.c.163.com", "", false)}
r, err := findBestMatchRegistry(image, registries)
require.NoError(t, err)
is.NotNil(r)
is.True(r.Authentication)
is.NoError(err, "")
is.NotNil(r, "")
is.True(r.Authentication, "")
is.Equal("docker.io", r.URL)
})
@@ -49,9 +47,9 @@ func TestFindBestMatchNeedAuthRegistry(t *testing.T) {
image := "portainer/portainer-ee:latest"
registries := []portainer.Registry{createNewRegistry("docker.io", "", true)}
r, err := findBestMatchRegistry(image, registries)
require.NoError(t, err)
is.NotNil(r)
is.True(r.Authentication)
is.NoError(err, "")
is.NotNil(r, "")
is.True(r.Authentication, "")
is.Equal("docker.io", r.URL)
})
}
-125
View File
@@ -1,125 +0,0 @@
package stats
import (
"context"
"errors"
"strings"
"sync"
"github.com/docker/docker/api/types/container"
)
type ContainerStats struct {
Running int `json:"running"`
Stopped int `json:"stopped"`
Healthy int `json:"healthy"`
Unhealthy int `json:"unhealthy"`
Total int `json:"total"`
}
type DockerClient interface {
ContainerInspect(ctx context.Context, containerID string) (container.InspectResponse, error)
}
func CalculateContainerStats(ctx context.Context, cli DockerClient, isSwarm bool, containers []container.Summary) (ContainerStats, error) {
if isSwarm {
return CalculateContainerStatsForSwarm(containers), nil
}
var running, stopped, healthy, unhealthy int
var mu sync.Mutex
var wg sync.WaitGroup
semaphore := make(chan struct{}, 5)
var aggErr error
var aggMu sync.Mutex
for i := range containers {
id := containers[i].ID
semaphore <- struct{}{}
wg.Go(func() {
defer func() { <-semaphore }()
containerInspection, err := cli.ContainerInspect(ctx, id)
stat := ContainerStats{}
if err != nil {
aggMu.Lock()
aggErr = errors.Join(aggErr, err)
aggMu.Unlock()
return
}
stat = getContainerStatus(containerInspection.State)
mu.Lock()
running += stat.Running
stopped += stat.Stopped
healthy += stat.Healthy
unhealthy += stat.Unhealthy
mu.Unlock()
})
}
wg.Wait()
return ContainerStats{
Running: running,
Stopped: stopped,
Healthy: healthy,
Unhealthy: unhealthy,
Total: len(containers),
}, aggErr
}
func getContainerStatus(state *container.State) ContainerStats {
stat := ContainerStats{}
if state == nil {
return stat
}
switch state.Status {
case container.StateRunning:
stat.Running++
case container.StateExited, container.StateDead:
stat.Stopped++
}
if state.Health != nil {
switch state.Health.Status {
case container.Healthy:
stat.Healthy++
case container.Unhealthy:
stat.Unhealthy++
}
}
return stat
}
// This is a temporary workaround to calculate container stats for Swarm
// TODO: Remove this once we have a proper way to calculate container stats for Swarm
func CalculateContainerStatsForSwarm(containers []container.Summary) ContainerStats {
var running, stopped, healthy, unhealthy int
for _, container := range containers {
switch container.State {
case "running":
running++
case "exited", "stopped":
stopped++
}
if strings.Contains(container.Status, "(healthy)") {
healthy++
} else if strings.Contains(container.Status, "(unhealthy)") {
unhealthy++
}
}
return ContainerStats{
Running: running,
Stopped: stopped,
Healthy: healthy,
Unhealthy: unhealthy,
Total: len(containers),
}
}
-253
View File
@@ -1,253 +0,0 @@
package stats
import (
"context"
"errors"
"testing"
"time"
"github.com/docker/docker/api/types/container"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
)
// MockDockerClient implements the DockerClient interface for testing
type MockDockerClient struct {
mock.Mock
}
func (m *MockDockerClient) ContainerInspect(ctx context.Context, containerID string) (container.InspectResponse, error) {
args := m.Called(ctx, containerID)
return args.Get(0).(container.InspectResponse), args.Error(1)
}
func TestCalculateContainerStats(t *testing.T) {
mockClient := new(MockDockerClient)
// Test containers - using enough containers to test concurrent processing
containers := []container.Summary{
{ID: "container1"},
{ID: "container2"},
{ID: "container3"},
{ID: "container4"},
{ID: "container5"},
{ID: "container6"},
{ID: "container7"},
{ID: "container8"},
{ID: "container9"},
{ID: "container10"},
}
// Setup mock expectations with different container states to test various scenarios
containerStates := []struct {
id string
status string
health *container.Health
expected ContainerStats
}{
{"container1", container.StateRunning, &container.Health{Status: container.Healthy}, ContainerStats{Running: 1, Stopped: 0, Healthy: 1, Unhealthy: 0}},
{"container2", container.StateRunning, &container.Health{Status: container.Unhealthy}, ContainerStats{Running: 1, Stopped: 0, Healthy: 0, Unhealthy: 1}},
{"container3", container.StateRunning, nil, ContainerStats{Running: 1, Stopped: 0, Healthy: 0, Unhealthy: 0}},
{"container4", container.StateExited, nil, ContainerStats{Running: 0, Stopped: 1, Healthy: 0, Unhealthy: 0}},
{"container5", container.StateDead, nil, ContainerStats{Running: 0, Stopped: 1, Healthy: 0, Unhealthy: 0}},
{"container6", container.StateRunning, &container.Health{Status: container.Healthy}, ContainerStats{Running: 1, Stopped: 0, Healthy: 1, Unhealthy: 0}},
{"container7", container.StateRunning, &container.Health{Status: container.Unhealthy}, ContainerStats{Running: 1, Stopped: 0, Healthy: 0, Unhealthy: 1}},
{"container8", container.StateExited, nil, ContainerStats{Running: 0, Stopped: 1, Healthy: 0, Unhealthy: 0}},
{"container9", container.StateRunning, nil, ContainerStats{Running: 1, Stopped: 0, Healthy: 0, Unhealthy: 0}},
{"container10", container.StateDead, nil, ContainerStats{Running: 0, Stopped: 1, Healthy: 0, Unhealthy: 0}},
}
expected := ContainerStats{}
// Setup mock expectations for all containers with artificial delays to simulate real Docker calls
for _, state := range containerStates {
mockClient.On("ContainerInspect", mock.Anything, state.id).Return(container.InspectResponse{
ContainerJSONBase: &container.ContainerJSONBase{
State: &container.State{
Status: state.status,
Health: state.health,
},
},
}, nil).After(50 * time.Millisecond) // Simulate 50ms Docker API call
expected.Running += state.expected.Running
expected.Stopped += state.expected.Stopped
expected.Healthy += state.expected.Healthy
expected.Unhealthy += state.expected.Unhealthy
expected.Total++
}
// Call the function and measure time
startTime := time.Now()
stats, err := CalculateContainerStats(context.Background(), mockClient, false, containers)
require.NoError(t, err, "failed to calculate container stats")
duration := time.Since(startTime)
// Assert results
assert.Equal(t, expected, stats)
assert.Equal(t, expected.Running, stats.Running)
assert.Equal(t, expected.Stopped, stats.Stopped)
assert.Equal(t, expected.Healthy, stats.Healthy)
assert.Equal(t, expected.Unhealthy, stats.Unhealthy)
assert.Equal(t, 10, stats.Total)
// Verify concurrent processing by checking that all mock calls were made
mockClient.AssertExpectations(t)
// Test concurrency: With 5 workers and 10 containers taking 50ms each:
// Sequential would take: 10 * 50ms = 500ms
sequentialTime := 10 * 50 * time.Millisecond
// Verify that concurrent processing is actually faster than sequential
// Allow some overhead for goroutine scheduling
assert.Less(t, duration, sequentialTime, "Concurrent processing should be faster than sequential")
// Concurrent should take: ~100-150ms (depending on scheduling)
assert.Less(t, duration, 150*time.Millisecond, "Concurrent processing should be significantly faster")
assert.Greater(t, duration, 100*time.Millisecond, "Concurrent processing should be longer than 100ms")
}
func TestCalculateContainerStatsAllErrors(t *testing.T) {
mockClient := new(MockDockerClient)
// Test containers
containers := []container.Summary{
{ID: "container1"},
{ID: "container2"},
}
// Setup mock expectations with all calls returning errors
mockClient.On("ContainerInspect", mock.Anything, "container1").Return(container.InspectResponse{}, errors.New("network error"))
mockClient.On("ContainerInspect", mock.Anything, "container2").Return(container.InspectResponse{}, errors.New("permission denied"))
// Call the function
stats, err := CalculateContainerStats(context.Background(), mockClient, false, containers)
// Assert that an error was returned
require.Error(t, err, "should return error when all containers fail to inspect")
assert.Contains(t, err.Error(), "network error", "error should contain one of the original error messages")
assert.Contains(t, err.Error(), "permission denied", "error should contain the other original error message")
// Assert that stats are zero since no containers were successfully processed
expectedStats := ContainerStats{
Running: 0,
Stopped: 0,
Healthy: 0,
Unhealthy: 0,
Total: 2, // total containers processed
}
assert.Equal(t, expectedStats, stats)
// Verify all mock calls were made
mockClient.AssertExpectations(t)
}
func TestGetContainerStatus(t *testing.T) {
testCases := []struct {
name string
state *container.State
expected ContainerStats
}{
{
name: "running healthy container",
state: &container.State{
Status: container.StateRunning,
Health: &container.Health{
Status: container.Healthy,
},
},
expected: ContainerStats{
Running: 1,
Stopped: 0,
Healthy: 1,
Unhealthy: 0,
},
},
{
name: "running unhealthy container",
state: &container.State{
Status: container.StateRunning,
Health: &container.Health{
Status: container.Unhealthy,
},
},
expected: ContainerStats{
Running: 1,
Stopped: 0,
Healthy: 0,
Unhealthy: 1,
},
},
{
name: "running container without health check",
state: &container.State{
Status: container.StateRunning,
},
expected: ContainerStats{
Running: 1,
Stopped: 0,
Healthy: 0,
Unhealthy: 0,
},
},
{
name: "exited container",
state: &container.State{
Status: container.StateExited,
},
expected: ContainerStats{
Running: 0,
Stopped: 1,
Healthy: 0,
Unhealthy: 0,
},
},
{
name: "dead container",
state: &container.State{
Status: container.StateDead,
},
expected: ContainerStats{
Running: 0,
Stopped: 1,
Healthy: 0,
Unhealthy: 0,
},
},
{
name: "nil state",
state: nil,
expected: ContainerStats{
Running: 0,
Stopped: 0,
Healthy: 0,
Unhealthy: 0,
},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
stat := getContainerStatus(testCase.state)
assert.Equal(t, testCase.expected, stat)
})
}
}
func TestCalculateContainerStatsForSwarm(t *testing.T) {
containers := []container.Summary{
{State: "running"},
{State: "running", Status: "Up 5 minutes (healthy)"},
{State: "exited"},
{State: "stopped"},
{State: "running", Status: "Up 10 minutes"},
{State: "running", Status: "Up about an hour (unhealthy)"},
}
stats := CalculateContainerStatsForSwarm(containers)
assert.Equal(t, 4, stats.Running)
assert.Equal(t, 2, stats.Stopped)
assert.Equal(t, 1, stats.Healthy)
assert.Equal(t, 1, stats.Unhealthy)
assert.Equal(t, 6, stats.Total)
}
+5
View File
@@ -49,6 +49,11 @@ type (
// Is relative path supported
SupportRelativePath bool
// AlwaysCloneGitRepoForRelativePath is a flag indicating if the agent must always clone the git repository for relative path.
// This field is only valid when SupportRelativePath is true.
// Used only for EE
AlwaysCloneGitRepoForRelativePath bool
// Mount point for relative path
FilesystemPath string
// Used only for EE
+2 -4
View File
@@ -8,9 +8,7 @@ import (
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_createEnvFile(t *testing.T) {
@@ -63,7 +61,7 @@ func Test_createEnvFile(t *testing.T) {
assert.Equal(t, tt.expected, string(content))
} else {
assert.Empty(t, result)
assert.Equal(t, "", result)
}
})
}
@@ -81,7 +79,7 @@ func Test_createEnvFile_mergesDefultAndInplaceEnvVars(t *testing.T) {
}
result, err := createEnvFile(stack)
assert.Equal(t, filepath.Join(stack.ProjectPath, "stack.env"), result)
require.NoError(t, err)
assert.NoError(t, err)
assert.FileExists(t, path.Join(dir, "stack.env"))
f, _ := os.Open(path.Join(dir, "stack.env"))
content, _ := io.ReadAll(f)
+7 -8
View File
@@ -7,7 +7,6 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type mockKubectlClient struct {
@@ -69,7 +68,7 @@ func TestExecuteKubectlOperation_Apply_Success(t *testing.T) {
manifests := []string{"manifest1.yaml", "manifest2.yaml"}
err := testExecuteKubectlOperation(mockClient, "apply", manifests)
require.NoError(t, err)
assert.NoError(t, err)
assert.True(t, called)
}
@@ -87,7 +86,7 @@ func TestExecuteKubectlOperation_Apply_Error(t *testing.T) {
manifests := []string{"error.yaml"}
err := testExecuteKubectlOperation(mockClient, "apply", manifests)
require.Error(t, err)
assert.Error(t, err)
assert.Contains(t, err.Error(), expectedErr.Error())
assert.True(t, called)
}
@@ -105,7 +104,7 @@ func TestExecuteKubectlOperation_Delete_Success(t *testing.T) {
manifests := []string{"manifest1.yaml"}
err := testExecuteKubectlOperation(mockClient, "delete", manifests)
require.NoError(t, err)
assert.NoError(t, err)
assert.True(t, called)
}
@@ -123,7 +122,7 @@ func TestExecuteKubectlOperation_Delete_Error(t *testing.T) {
manifests := []string{"error.yaml"}
err := testExecuteKubectlOperation(mockClient, "delete", manifests)
require.Error(t, err)
assert.Error(t, err)
assert.Contains(t, err.Error(), expectedErr.Error())
assert.True(t, called)
}
@@ -141,7 +140,7 @@ func TestExecuteKubectlOperation_RolloutRestart_Success(t *testing.T) {
resources := []string{"deployment/nginx"}
err := testExecuteKubectlOperation(mockClient, "rollout-restart", resources)
require.NoError(t, err)
assert.NoError(t, err)
assert.True(t, called)
}
@@ -159,7 +158,7 @@ func TestExecuteKubectlOperation_RolloutRestart_Error(t *testing.T) {
resources := []string{"deployment/error"}
err := testExecuteKubectlOperation(mockClient, "rollout-restart", resources)
require.Error(t, err)
assert.Error(t, err)
assert.Contains(t, err.Error(), expectedErr.Error())
assert.True(t, called)
}
@@ -169,6 +168,6 @@ func TestExecuteKubectlOperation_UnsupportedOperation(t *testing.T) {
err := testExecuteKubectlOperation(mockClient, "unsupported", []string{})
require.Error(t, err)
assert.Error(t, err)
assert.Contains(t, err.Error(), "unsupported operation")
}
+8 -9
View File
@@ -7,13 +7,12 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_copyFile_returnsError_whenSourceDoesNotExist(t *testing.T) {
tmpdir := t.TempDir()
err := copyFile("does-not-exist", tmpdir)
require.Error(t, err)
assert.Error(t, err)
}
func Test_copyFile_shouldMakeAbackup(t *testing.T) {
@@ -22,7 +21,7 @@ func Test_copyFile_shouldMakeAbackup(t *testing.T) {
os.WriteFile(path.Join(tmpdir, "origin"), content, 0600)
err := copyFile(path.Join(tmpdir, "origin"), path.Join(tmpdir, "copy"))
require.NoError(t, err)
assert.NoError(t, err)
copyContent, _ := os.ReadFile(path.Join(tmpdir, "copy"))
assert.Equal(t, content, copyContent)
@@ -31,7 +30,7 @@ func Test_copyFile_shouldMakeAbackup(t *testing.T) {
func Test_CopyDir_shouldCopyAllFilesAndDirectories(t *testing.T) {
destination := t.TempDir()
err := CopyDir("./testdata/copy_test", destination, true)
require.NoError(t, err)
assert.NoError(t, err)
assert.FileExists(t, filepath.Join(destination, "copy_test", "outer"))
assert.FileExists(t, filepath.Join(destination, "copy_test", "dir", ".dotfile"))
@@ -41,7 +40,7 @@ func Test_CopyDir_shouldCopyAllFilesAndDirectories(t *testing.T) {
func Test_CopyDir_shouldCopyOnlyDirContents(t *testing.T) {
destination := t.TempDir()
err := CopyDir("./testdata/copy_test", destination, false)
require.NoError(t, err)
assert.NoError(t, err)
assert.FileExists(t, filepath.Join(destination, "outer"))
assert.FileExists(t, filepath.Join(destination, "dir", ".dotfile"))
@@ -51,7 +50,7 @@ func Test_CopyDir_shouldCopyOnlyDirContents(t *testing.T) {
func Test_CopyPath_shouldSkipWhenNotExist(t *testing.T) {
tmpdir := t.TempDir()
err := CopyPath("does-not-exists", tmpdir)
require.NoError(t, err)
assert.NoError(t, err)
assert.NoFileExists(t, tmpdir)
}
@@ -63,17 +62,17 @@ func Test_CopyPath_shouldCopyFile(t *testing.T) {
os.MkdirAll(path.Join(tmpdir, "backup"), 0700)
err := CopyPath(path.Join(tmpdir, "file"), path.Join(tmpdir, "backup"))
require.NoError(t, err)
assert.NoError(t, err)
copyContent, err := os.ReadFile(path.Join(tmpdir, "backup", "file"))
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, content, copyContent)
}
func Test_CopyPath_shouldCopyDir(t *testing.T) {
destination := t.TempDir()
err := CopyPath("./testdata/copy_test", destination)
require.NoError(t, err)
assert.NoError(t, err)
assert.FileExists(t, filepath.Join(destination, "copy_test", "outer"))
assert.FileExists(t, filepath.Join(destination, "copy_test", "dir", ".dotfile"))
+4 -5
View File
@@ -8,7 +8,6 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_fileSystemService_FileExists_whenFileExistsShouldReturnTrue(t *testing.T) {
@@ -31,14 +30,14 @@ func Test_FileExists_whenFileNotExistsShouldReturnFalse(t *testing.T) {
func testHelperFileExists_fileExists(t *testing.T, checker func(path string) (bool, error)) {
file, err := os.CreateTemp("", t.Name())
require.NoError(t, err, "CreateTemp should not fail")
assert.NoError(t, err, "CreateTemp should not fail")
t.Cleanup(func() {
os.RemoveAll(file.Name())
})
exists, err := checker(file.Name())
require.NoError(t, err, "FileExists should not fail")
assert.NoError(t, err, "FileExists should not fail")
assert.True(t, exists)
}
@@ -47,10 +46,10 @@ func testHelperFileExists_fileNotExists(t *testing.T, checker func(path string)
filePath := path.Join(t.TempDir(), fmt.Sprintf("%s%d", t.Name(), rand.Int()))
err := os.RemoveAll(filePath)
require.NoError(t, err, "RemoveAll should not fail")
assert.NoError(t, err, "RemoveAll should not fail")
exists, err := checker(filePath)
require.NoError(t, err, "FileExists should not fail")
assert.NoError(t, err, "FileExists should not fail")
assert.False(t, exists)
}
+20 -23
View File
@@ -6,7 +6,6 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
var content = []byte("content")
@@ -14,25 +13,25 @@ var content = []byte("content")
func Test_movePath_shouldFailIfSourceDirDoesNotExist(t *testing.T) {
sourceDir := "missing"
destinationDir := t.TempDir()
file1 := addFile(t, destinationDir, "dir", "file")
file2 := addFile(t, destinationDir, "file")
file1 := addFile(destinationDir, "dir", "file")
file2 := addFile(destinationDir, "file")
err := MoveDirectory(sourceDir, destinationDir, false)
require.Error(t, err, "move directory should fail when source path is missing")
assert.Error(t, err, "move directory should fail when source path is missing")
assert.FileExists(t, file1, "destination dir contents should remain")
assert.FileExists(t, file2, "destination dir contents should remain")
}
func Test_movePath_shouldFailIfDestinationDirExists(t *testing.T) {
sourceDir := t.TempDir()
file1 := addFile(t, sourceDir, "dir", "file")
file2 := addFile(t, sourceDir, "file")
file1 := addFile(sourceDir, "dir", "file")
file2 := addFile(sourceDir, "file")
destinationDir := t.TempDir()
file3 := addFile(t, destinationDir, "dir", "file")
file4 := addFile(t, destinationDir, "file")
file3 := addFile(destinationDir, "dir", "file")
file4 := addFile(destinationDir, "file")
err := MoveDirectory(sourceDir, destinationDir, false)
require.Error(t, err, "move directory should fail when destination directory already exists")
assert.Error(t, err, "move directory should fail when destination directory already exists")
assert.FileExists(t, file1, "source dir contents should remain")
assert.FileExists(t, file2, "source dir contents should remain")
assert.FileExists(t, file3, "destination dir contents should remain")
@@ -41,14 +40,14 @@ func Test_movePath_shouldFailIfDestinationDirExists(t *testing.T) {
func Test_movePath_succesIfOverwriteSetWhenDestinationDirExists(t *testing.T) {
sourceDir := t.TempDir()
file1 := addFile(t, sourceDir, "dir", "file")
file2 := addFile(t, sourceDir, "file")
file1 := addFile(sourceDir, "dir", "file")
file2 := addFile(sourceDir, "file")
destinationDir := t.TempDir()
file3 := addFile(t, destinationDir, "dir", "file")
file4 := addFile(t, destinationDir, "file")
file3 := addFile(destinationDir, "dir", "file")
file4 := addFile(destinationDir, "file")
err := MoveDirectory(sourceDir, destinationDir, true)
require.NoError(t, err)
assert.NoError(t, err)
assert.NoFileExists(t, file1, "source dir contents should be moved")
assert.NoFileExists(t, file2, "source dir contents should be moved")
assert.FileExists(t, file3, "destination dir contents should remain")
@@ -59,34 +58,32 @@ func Test_movePath_successWhenSourceExistsAndDestinationIsMissing(t *testing.T)
tmp := t.TempDir()
sourceDir := path.Join(tmp, "source")
os.Mkdir(sourceDir, 0766)
file1 := addFile(t, sourceDir, "dir", "file")
file2 := addFile(t, sourceDir, "file")
file1 := addFile(sourceDir, "dir", "file")
file2 := addFile(sourceDir, "file")
destinationDir := path.Join(tmp, "destination")
err := MoveDirectory(sourceDir, destinationDir, false)
require.NoError(t, err)
assert.NoError(t, err)
assert.NoFileExists(t, file1, "source dir contents should be moved")
assert.NoFileExists(t, file2, "source dir contents should be moved")
assertFileContent(t, path.Join(destinationDir, "file"))
assertFileContent(t, path.Join(destinationDir, "dir", "file"))
}
func addFile(t *testing.T, fileParts ...string) (filepath string) {
func addFile(fileParts ...string) (filepath string) {
if len(fileParts) > 2 {
dir := path.Join(fileParts[:len(fileParts)-1]...)
err := os.MkdirAll(dir, 0766)
require.NoError(t, err)
os.MkdirAll(dir, 0766)
}
p := path.Join(fileParts...)
err := os.WriteFile(p, content, 0766)
require.NoError(t, err)
os.WriteFile(p, content, 0766)
return p
}
func assertFileContent(t *testing.T, filePath string) {
actualContent, err := os.ReadFile(filePath)
require.NoError(t, err, "failed to read file %s", filePath)
assert.NoErrorf(t, err, "failed to read file %s", filePath)
assert.Equal(t, content, actualContent, "file %s content doesn't match", filePath)
}
+2 -2
View File
@@ -5,14 +5,14 @@ import (
"path"
"testing"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/assert"
)
func createService(t *testing.T) *Service {
dataStorePath := path.Join(t.TempDir(), t.Name())
service, err := NewService(dataStorePath, "")
require.NoError(t, err, "NewService should not fail")
assert.NoError(t, err, "NewService should not fail")
t.Cleanup(func() {
os.RemoveAll(dataStorePath)
+4 -5
View File
@@ -6,7 +6,6 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_WriteFile_CanStoreContentInANewFile(t *testing.T) {
@@ -15,7 +14,7 @@ func Test_WriteFile_CanStoreContentInANewFile(t *testing.T) {
content := []byte("content")
err := WriteToFile(tmpFilePath, content)
require.NoError(t, err)
assert.NoError(t, err)
fileContent, _ := os.ReadFile(tmpFilePath)
assert.Equal(t, content, fileContent)
@@ -26,11 +25,11 @@ func Test_WriteFile_CanOverwriteExistingFile(t *testing.T) {
tmpFilePath := path.Join(tmpDir, "dummy")
err := WriteToFile(tmpFilePath, []byte("content"))
require.NoError(t, err)
assert.NoError(t, err)
content := []byte("new content")
err = WriteToFile(tmpFilePath, content)
require.NoError(t, err)
assert.NoError(t, err)
fileContent, _ := os.ReadFile(tmpFilePath)
assert.Equal(t, content, fileContent)
@@ -42,7 +41,7 @@ func Test_WriteFile_CanWriteANestedPath(t *testing.T) {
content := []byte("content")
err := WriteToFile(tmpFilePath, content)
require.NoError(t, err)
assert.NoError(t, err)
fileContent, _ := os.ReadFile(tmpFilePath)
assert.Equal(t, content, fileContent)
+7 -8
View File
@@ -12,7 +12,6 @@ import (
_ "github.com/joho/godotenv/autoload"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const privateAzureRepoURL = "https://portainer.visualstudio.com/gitops-test/_git/gitops-test"
@@ -68,7 +67,7 @@ func TestService_ClonePublicRepository_Azure(t *testing.T) {
gittypes.GitCredentialAuthType_Basic,
false,
)
require.NoError(t, err)
assert.NoError(t, err)
assert.FileExists(t, filepath.Join(dst, "README.md"))
})
}
@@ -91,7 +90,7 @@ func TestService_ClonePrivateRepository_Azure(t *testing.T) {
gittypes.GitCredentialAuthType_Basic,
false,
)
require.NoError(t, err)
assert.NoError(t, err)
assert.FileExists(t, filepath.Join(dst, "README.md"))
}
@@ -109,7 +108,7 @@ func TestService_LatestCommitID_Azure(t *testing.T) {
gittypes.GitCredentialAuthType_Basic,
false,
)
require.NoError(t, err)
assert.NoError(t, err)
assert.NotEmpty(t, id, "cannot guarantee commit id, but it should be not empty")
}
@@ -128,7 +127,7 @@ func TestService_ListRefs_Azure(t *testing.T) {
false,
false,
)
require.NoError(t, err)
assert.NoError(t, err)
assert.GreaterOrEqual(t, len(refs), 1)
}
@@ -290,14 +289,14 @@ func TestService_ListFiles_Azure(t *testing.T) {
false,
)
if tt.expect.shouldFail {
require.Error(t, err)
assert.Error(t, err)
if tt.expect.err != nil {
assert.Equal(t, tt.expect.err, err)
}
} else {
require.NoError(t, err)
assert.NoError(t, err)
if tt.expect.matchedCount > 0 {
assert.NotEmpty(t, paths)
assert.Greater(t, len(paths), 0)
}
}
})
+18 -23
View File
@@ -9,9 +9,7 @@ import (
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/portainer/portainer/pkg/fips"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_buildDownloadUrl(t *testing.T) {
@@ -21,18 +19,15 @@ func Test_buildDownloadUrl(t *testing.T) {
project: "project",
repository: "repository",
}, "refs/heads/main")
require.NoError(t, err)
expectedUrl, err := url.Parse("https://dev.azure.com/organisation/project/_apis/git/repositories/repository/items?scopePath=/&download=true&versionDescriptor.version=main&$format=zip&recursionLevel=full&api-version=6.0&versionDescriptor.versionType=branch")
require.NoError(t, err)
actualUrl, err := url.Parse(u)
require.NoError(t, err)
assert.Equal(t, expectedUrl.Host, actualUrl.Host)
assert.Equal(t, expectedUrl.Scheme, actualUrl.Scheme)
assert.Equal(t, expectedUrl.Path, actualUrl.Path)
assert.Equal(t, expectedUrl.Query(), actualUrl.Query())
expectedUrl, _ := url.Parse("https://dev.azure.com/organisation/project/_apis/git/repositories/repository/items?scopePath=/&download=true&versionDescriptor.version=main&$format=zip&recursionLevel=full&api-version=6.0&versionDescriptor.versionType=branch")
actualUrl, _ := url.Parse(u)
if assert.NoError(t, err) {
assert.Equal(t, expectedUrl.Host, actualUrl.Host)
assert.Equal(t, expectedUrl.Scheme, actualUrl.Scheme)
assert.Equal(t, expectedUrl.Path, actualUrl.Path)
assert.Equal(t, expectedUrl.Query(), actualUrl.Query())
}
}
func Test_buildRootItemUrl(t *testing.T) {
@@ -45,7 +40,7 @@ func Test_buildRootItemUrl(t *testing.T) {
expectedUrl, _ := url.Parse("https://dev.azure.com/organisation/project/_apis/git/repositories/repository/items?scopePath=/&api-version=6.0&versionDescriptor.version=main&versionDescriptor.versionType=branch")
actualUrl, _ := url.Parse(u)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, expectedUrl.Host, actualUrl.Host)
assert.Equal(t, expectedUrl.Scheme, actualUrl.Scheme)
assert.Equal(t, expectedUrl.Path, actualUrl.Path)
@@ -62,7 +57,7 @@ func Test_buildRefsUrl(t *testing.T) {
expectedUrl, _ := url.Parse("https://dev.azure.com/organisation/project/_apis/git/repositories/repository/refs?api-version=6.0")
actualUrl, _ := url.Parse(u)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, expectedUrl.Host, actualUrl.Host)
assert.Equal(t, expectedUrl.Scheme, actualUrl.Scheme)
assert.Equal(t, expectedUrl.Path, actualUrl.Path)
@@ -79,7 +74,7 @@ func Test_buildTreeUrl(t *testing.T) {
expectedUrl, _ := url.Parse("https://dev.azure.com/organisation/project/_apis/git/repositories/repository/trees/sha1?api-version=6.0&recursive=true")
actualUrl, _ := url.Parse(u)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, expectedUrl.Host, actualUrl.Host)
assert.Equal(t, expectedUrl.Scheme, actualUrl.Scheme)
assert.Equal(t, expectedUrl.Path, actualUrl.Path)
@@ -309,7 +304,7 @@ func Test_azureDownloader_downloadZipFromAzureDevOps(t *testing.T) {
},
}
_, err := a.downloadZipFromAzureDevOps(context.Background(), option)
require.Error(t, err)
assert.Error(t, err)
assert.Equal(t, tt.want, zipRequestAuth)
})
}
@@ -507,12 +502,12 @@ func Test_listRefs_azure(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
refs, err := client.listRefs(context.TODO(), tt.args)
if tt.expect.err == nil {
require.NoError(t, err)
assert.NoError(t, err)
if tt.expect.refsCount > 0 {
assert.NotEmpty(t, refs)
assert.Greater(t, len(refs), 0)
}
} else {
require.Error(t, err)
assert.Error(t, err)
assert.Equal(t, tt.expect.err, err)
}
})
@@ -618,14 +613,14 @@ func Test_listFiles_azure(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
paths, err := client.listFiles(context.TODO(), tt.args)
if tt.expect.shouldFail {
require.Error(t, err)
assert.Error(t, err)
if tt.expect.err != nil {
assert.Equal(t, tt.expect.err, err)
}
} else {
require.NoError(t, err)
assert.NoError(t, err)
if tt.expect.matchedCount > 0 {
assert.NotEmpty(t, paths)
assert.Greater(t, len(paths), 0)
}
}
})
+15 -17
View File
@@ -9,9 +9,7 @@ import (
"time"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const (
@@ -37,7 +35,7 @@ func TestService_ClonePrivateRepository_GitHub(t *testing.T) {
gittypes.GitCredentialAuthType_Basic,
false,
)
require.NoError(t, err)
assert.NoError(t, err)
assert.FileExists(t, filepath.Join(dst, "README.md"))
}
@@ -57,7 +55,7 @@ func TestService_LatestCommitID_GitHub(t *testing.T) {
gittypes.GitCredentialAuthType_Basic,
false,
)
require.NoError(t, err)
assert.NoError(t, err)
assert.NotEmpty(t, id, "cannot guarantee commit id, but it should be not empty")
}
@@ -70,7 +68,7 @@ func TestService_ListRefs_GitHub(t *testing.T) {
repositoryUrl := privateGitRepoURL
refs, err := service.ListRefs(repositoryUrl, username, accessToken, gittypes.GitCredentialAuthType_Basic, false, false)
require.NoError(t, err)
assert.NoError(t, err)
assert.GreaterOrEqual(t, len(refs), 1)
}
@@ -233,14 +231,14 @@ func TestService_ListFiles_GitHub(t *testing.T) {
false,
)
if tt.expect.shouldFail {
require.Error(t, err)
assert.Error(t, err)
if tt.expect.err != nil {
assert.Equal(t, tt.expect.err, err)
}
} else {
require.NoError(t, err)
assert.NoError(t, err)
if tt.expect.matchedCount > 0 {
assert.NotEmpty(t, paths)
assert.Greater(t, len(paths), 0)
}
}
})
@@ -363,12 +361,12 @@ func TestService_HardRefresh_ListRefs_GitHub(t *testing.T) {
repositoryUrl := privateGitRepoURL
refs, err := service.ListRefs(repositoryUrl, username, accessToken, gittypes.GitCredentialAuthType_Basic, false, false)
require.NoError(t, err)
assert.NoError(t, err)
assert.GreaterOrEqual(t, len(refs), 1)
assert.Equal(t, 1, service.repoRefCache.Len())
_, err = service.ListRefs(repositoryUrl, username, "fake-token", gittypes.GitCredentialAuthType_Basic, false, false)
require.Error(t, err)
assert.Error(t, err)
assert.Equal(t, 1, service.repoRefCache.Len())
}
@@ -381,7 +379,7 @@ func TestService_HardRefresh_ListRefs_And_RemoveAllCaches_GitHub(t *testing.T) {
repositoryUrl := privateGitRepoURL
refs, err := service.ListRefs(repositoryUrl, username, accessToken, gittypes.GitCredentialAuthType_Basic, false, false)
require.NoError(t, err)
assert.NoError(t, err)
assert.GreaterOrEqual(t, len(refs), 1)
assert.Equal(t, 1, service.repoRefCache.Len())
@@ -396,7 +394,7 @@ func TestService_HardRefresh_ListRefs_And_RemoveAllCaches_GitHub(t *testing.T) {
[]string{},
false,
)
require.NoError(t, err)
assert.NoError(t, err)
assert.GreaterOrEqual(t, len(files), 1)
assert.Equal(t, 1, service.repoFileCache.Len())
@@ -411,16 +409,16 @@ func TestService_HardRefresh_ListRefs_And_RemoveAllCaches_GitHub(t *testing.T) {
[]string{},
false,
)
require.NoError(t, err)
assert.NoError(t, err)
assert.GreaterOrEqual(t, len(files), 1)
assert.Equal(t, 2, service.repoFileCache.Len())
_, err = service.ListRefs(repositoryUrl, username, "fake-token", gittypes.GitCredentialAuthType_Basic, false, false)
require.Error(t, err)
assert.Error(t, err)
assert.Equal(t, 1, service.repoRefCache.Len())
_, err = service.ListRefs(repositoryUrl, username, "fake-token", gittypes.GitCredentialAuthType_Basic, true, false)
require.Error(t, err)
assert.Error(t, err)
assert.Equal(t, 1, service.repoRefCache.Len())
// The relevant file caches should be removed too
assert.Equal(t, 0, service.repoFileCache.Len())
@@ -444,7 +442,7 @@ func TestService_HardRefresh_ListFiles_GitHub(t *testing.T) {
[]string{},
false,
)
require.NoError(t, err)
assert.NoError(t, err)
assert.GreaterOrEqual(t, len(files), 1)
assert.Equal(t, 1, service.repoFileCache.Len())
@@ -459,7 +457,7 @@ func TestService_HardRefresh_ListFiles_GitHub(t *testing.T) {
[]string{},
false,
)
require.Error(t, err)
assert.Error(t, err)
assert.Equal(t, 0, service.repoFileCache.Len())
}
+12 -13
View File
@@ -13,7 +13,6 @@ import (
"github.com/go-git/go-git/v5/plumbing/object"
"github.com/pkg/errors"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func setup(t *testing.T) string {
@@ -40,7 +39,7 @@ func Test_ClonePublicRepository_Shallow(t *testing.T) {
dir := t.TempDir()
t.Logf("Cloning into %s", dir)
err := service.CloneRepository(dir, repositoryURL, referenceName, "", "", gittypes.GitCredentialAuthType_Basic, false)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, 1, getCommitHistoryLength(t, dir), "cloned repo has incorrect depth")
}
@@ -52,7 +51,7 @@ func Test_ClonePublicRepository_NoGitDirectory(t *testing.T) {
dir := t.TempDir()
t.Logf("Cloning into %s", dir)
err := service.CloneRepository(dir, repositoryURL, referenceName, "", "", gittypes.GitCredentialAuthType_Basic, false)
require.NoError(t, err)
assert.NoError(t, err)
assert.NoDirExists(t, filepath.Join(dir, ".git"))
}
@@ -75,7 +74,7 @@ func Test_cloneRepository(t *testing.T) {
depth: 10,
})
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, 4, getCommitHistoryLength(t, dir), "cloned repo has incorrect depth")
}
@@ -87,7 +86,7 @@ func Test_latestCommitID(t *testing.T) {
id, err := service.LatestCommitID(repositoryURL, referenceName, "", "", gittypes.GitCredentialAuthType_Basic, false)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, "68dcaa7bd452494043c64252ab90db0f98ecf8d2", id)
}
@@ -98,7 +97,7 @@ func Test_ListRefs(t *testing.T) {
fs, err := service.ListRefs(repositoryURL, "", "", gittypes.GitCredentialAuthType_Basic, false, false)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, []string{"refs/heads/main"}, fs)
}
@@ -120,7 +119,7 @@ func Test_ListFiles(t *testing.T) {
false,
)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, []string{"docker-compose.yml"}, fs)
}
@@ -215,12 +214,12 @@ func Test_listRefsPrivateRepository(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
refs, err := client.listRefs(context.TODO(), tt.args)
if tt.expect.err == nil {
require.NoError(t, err)
assert.NoError(t, err)
if tt.expect.refsCount > 0 {
assert.NotEmpty(t, refs)
assert.Greater(t, len(refs), 0)
}
} else {
require.Error(t, err)
assert.Error(t, err)
assert.Equal(t, tt.expect.err, err)
}
})
@@ -326,14 +325,14 @@ func Test_listFilesPrivateRepository(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
paths, err := client.listFiles(context.TODO(), tt.args)
if tt.expect.shouldFail {
require.Error(t, err)
assert.Error(t, err)
if tt.expect.err != nil {
assert.Equal(t, tt.expect.err, err)
}
} else {
require.NoError(t, err)
assert.NoError(t, err)
if tt.expect.matchedCount > 0 {
assert.NotEmpty(t, paths)
assert.Greater(t, len(paths), 0)
}
}
})
+20
View File
@@ -0,0 +1,20 @@
package errors
import (
"errors"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
)
func TxResponse(err error, validResponse func() *httperror.HandlerError) *httperror.HandlerError {
if err != nil {
var handlerError *httperror.HandlerError
if errors.As(err, &handlerError) {
return handlerError
}
return httperror.InternalServerError("Unexpected error", err)
}
return validResponse()
}
+19 -18
View File
@@ -18,7 +18,6 @@ import (
"github.com/portainer/portainer/api/internal/testhelpers"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_restoreArchive_usingCombinationOfPasswords(t *testing.T) {
@@ -65,12 +64,13 @@ func Test_restoreArchive_usingCombinationOfPasswords(t *testing.T) {
adminMonitor,
)
// backup
//backup
archive := backup(t, h, test.backupPassword)
// restore
//restore
w := httptest.NewRecorder()
r := prepareMultipartRequest(t, test.restorePassword, archive)
r, err := prepareMultipartRequest(test.restorePassword, archive)
assert.Nil(t, err, "Shouldn't fail to write multipart form")
restoreErr := h.restore(w, r)
assert.Equal(t, test.fails, restoreErr != nil, "Didn't meet expectation of failing restore handler")
@@ -96,12 +96,13 @@ func Test_restoreArchive_shouldFailIfSystemWasAlreadyInitialized(t *testing.T) {
adminMonitor,
)
// backup
//backup
archive := backup(t, h, "password")
// restore
//restore
w := httptest.NewRecorder()
r := prepareMultipartRequest(t, "password", archive)
r, err := prepareMultipartRequest("password", archive)
assert.Nil(t, err, "Shouldn't fail to write multipart form")
restoreErr := h.restore(w, r)
assert.NotNil(t, restoreErr, "Should fail, because system it already initialized")
@@ -116,31 +117,31 @@ func backup(t *testing.T, h *Handler, password string) []byte {
assert.Nil(t, backupErr, "Backup should not fail")
response := w.Result()
archive, err := io.ReadAll(response.Body)
require.NoError(t, err)
archive, _ := io.ReadAll(response.Body)
response.Body.Close()
return archive
}
func prepareMultipartRequest(t *testing.T, password string, file []byte) *http.Request {
func prepareMultipartRequest(password string, file []byte) (*http.Request, error) {
var body bytes.Buffer
w := multipart.NewWriter(&body)
err := w.WriteField("password", password)
require.NoError(t, err)
if err != nil {
return nil, err
}
fw, err := w.CreateFormFile("file", "filename")
require.NoError(t, err)
_, err = io.Copy(fw, bytes.NewReader(file))
require.NoError(t, err)
if err != nil {
return nil, err
}
io.Copy(fw, bytes.NewReader(file))
r := httptest.NewRequest(http.MethodPost, "http://localhost/", &body)
r.Header.Set("Content-Type", w.FormDataContentType())
err = w.Close()
require.NoError(t, err)
w.Close()
return r
return r, nil
}
@@ -9,6 +9,7 @@ import (
"net/http/httptest"
"os"
"path/filepath"
"sync"
"testing"
"time"
@@ -24,8 +25,6 @@ import (
"github.com/segmentio/encoding/json"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.org/x/sync/errgroup"
)
func init() {
@@ -113,14 +112,15 @@ func createTestFile(targetPath string) error {
}
func prepareTestFolder(projectPath, filename string) error {
if err := os.MkdirAll(projectPath, fs.ModePerm); err != nil {
err := os.MkdirAll(projectPath, fs.ModePerm)
if err != nil {
return err
}
return createTestFile(filepath.Join(projectPath, filename))
}
func singleAPIRequest(h *Handler, jwt string, expect string) error {
func singleAPIRequest(h *Handler, jwt string, is *assert.Assertions, expect string) {
type response struct {
FileContent string
}
@@ -131,25 +131,15 @@ func singleAPIRequest(h *Handler, jwt string, expect string) error {
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
return errors.New("unexpected status code: " + http.StatusText(rr.Code))
}
is.Equal(http.StatusOK, rr.Code)
body, err := io.ReadAll(rr.Body)
if err != nil {
return err
}
is.NoError(err, "ReadAll should not return error")
var resp response
if err := json.Unmarshal(body, &resp); err != nil {
return err
}
if resp.FileContent != expect {
return errors.New("unexpected file content: " + resp.FileContent + ", expected: " + expect)
}
return nil
err = json.Unmarshal(body, &resp)
is.NoError(err, "response should be list json")
is.Equal(resp.FileContent, expect)
}
func Test_customTemplateGitFetch(t *testing.T) {
@@ -160,29 +150,28 @@ func Test_customTemplateGitFetch(t *testing.T) {
// create user(s)
user1 := &portainer.User{ID: 1, Username: "user-1", Role: portainer.StandardUserRole, PortainerAuthorizations: authorization.DefaultPortainerAuthorizations()}
err := store.User().Create(user1)
require.NoError(t, err, "error creating user 1")
is.NoError(err, "error creating user 1")
user2 := &portainer.User{ID: 2, Username: "user-2", Role: portainer.StandardUserRole, PortainerAuthorizations: authorization.DefaultPortainerAuthorizations()}
err = store.User().Create(user2)
require.NoError(t, err, "error creating user 2")
is.NoError(err, "error creating user 2")
dir, err := os.Getwd()
require.NoError(t, err, "error to get working directory")
is.NoError(err, "error to get working directory")
template1 := &portainer.CustomTemplate{ID: 1, Title: "custom-template-1", ProjectPath: filepath.Join(dir, "fixtures/custom_template_1"), GitConfig: &gittypes.RepoConfig{ConfigFilePath: "test-config-path.txt"}}
err = store.CustomTemplateService.Create(template1)
require.NoError(t, err, "error creating custom template 1")
is.NoError(err, "error creating custom template 1")
// prepare testing folder
err = prepareTestFolder(template1.ProjectPath, template1.GitConfig.ConfigFilePath)
require.NoError(t, err, "error creating testing folder")
is.NoError(err, "error creating testing folder")
defer os.RemoveAll(filepath.Join(dir, "fixtures"))
// setup services
jwtService, err := jwt.NewService("1h", store)
require.NoError(t, err, "Error initiating jwt service")
is.NoError(err, "Error initiating jwt service")
requestBouncer := security.NewRequestBouncer(store, jwtService, nil)
gitService := &TestGitService{
@@ -193,55 +182,52 @@ func Test_customTemplateGitFetch(t *testing.T) {
h := NewHandler(requestBouncer, store, fileService, gitService)
// generate two standard users' tokens
jwt1, _, err := jwtService.GenerateToken(&portainer.TokenData{ID: user1.ID, Username: user1.Username, Role: user1.Role})
require.NoError(t, err)
jwt2, _, err := jwtService.GenerateToken(&portainer.TokenData{ID: user2.ID, Username: user2.Username, Role: user2.Role})
require.NoError(t, err)
jwt1, _, _ := jwtService.GenerateToken(&portainer.TokenData{ID: user1.ID, Username: user1.Username, Role: user1.Role})
jwt2, _, _ := jwtService.GenerateToken(&portainer.TokenData{ID: user2.ID, Username: user2.Username, Role: user2.Role})
t.Run("can return the expected file content by a single call from one user", func(t *testing.T) {
err := singleAPIRequest(h, jwt1, "abcdefg")
require.NoError(t, err)
singleAPIRequest(h, jwt1, is, "abcdefg")
})
t.Run("can return the expected file content by multiple calls from one user", func(t *testing.T) {
var g errgroup.Group
var wg sync.WaitGroup
wg.Add(5)
for range 5 {
g.Go(func() error {
return singleAPIRequest(h, jwt1, "abcdefg")
})
go func() {
singleAPIRequest(h, jwt1, is, "abcdefg")
wg.Done()
}()
}
err := g.Wait()
require.NoError(t, err)
wg.Wait()
})
t.Run("can return the expected file content by multiple calls from different users", func(t *testing.T) {
var g errgroup.Group
var wg sync.WaitGroup
wg.Add(10)
for i := range 10 {
g.Go(func() error {
if i%2 == 0 {
return singleAPIRequest(h, jwt1, "abcdefg")
go func(j int) {
if j%2 == 0 {
singleAPIRequest(h, jwt1, is, "abcdefg")
} else {
singleAPIRequest(h, jwt2, is, "abcdefg")
}
return singleAPIRequest(h, jwt2, "abcdefg")
})
wg.Done()
}(i)
}
err := g.Wait()
require.NoError(t, err)
wg.Wait()
})
t.Run("can return the expected file content after a new commit is made", func(t *testing.T) {
err := singleAPIRequest(h, jwt1, "abcdefg")
require.NoError(t, err)
singleAPIRequest(h, jwt1, is, "abcdefg")
testFileContent = "gfedcba"
err = singleAPIRequest(h, jwt2, "gfedcba")
require.NoError(t, err)
singleAPIRequest(h, jwt2, is, "gfedcba")
})
t.Run("restore git repository if it is failed to download the new git repository", func(t *testing.T) {
@@ -260,11 +246,11 @@ func Test_customTemplateGitFetch(t *testing.T) {
var errResp httperror.HandlerError
err = json.NewDecoder(rr.Body).Decode(&errResp)
require.NoError(t, err, "failed to parse error body")
assert.NoError(t, err, "failed to parse error body")
assert.FileExists(t, gitService.targetFilePath, "previous git repository is not restored")
fileContent, err := os.ReadFile(gitService.targetFilePath)
require.NoError(t, err, "failed to read target file")
assert.NoError(t, err, "failed to read target file")
assert.Equal(t, "gfedcba", string(fileContent))
})
}
+12 -14
View File
@@ -11,7 +11,8 @@ import (
"github.com/docker/docker/api/types/volume"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/docker/stats"
"github.com/portainer/portainer/api/docker"
"github.com/portainer/portainer/api/http/errors"
"github.com/portainer/portainer/api/http/handler/docker/utils"
"github.com/portainer/portainer/api/http/middlewares"
"github.com/portainer/portainer/api/http/security"
@@ -25,12 +26,12 @@ type imagesCounters struct {
}
type dashboardResponse struct {
Containers stats.ContainerStats `json:"containers"`
Services int `json:"services"`
Images imagesCounters `json:"images"`
Volumes int `json:"volumes"`
Networks int `json:"networks"`
Stacks int `json:"stacks"`
Containers docker.ContainerStats `json:"containers"`
Services int `json:"services"`
Images imagesCounters `json:"images"`
Volumes int `json:"volumes"`
Networks int `json:"networks"`
Stacks int `json:"stacks"`
}
// @id dockerDashboard
@@ -143,18 +144,13 @@ func (h *Handler) dashboard(w http.ResponseWriter, r *http.Request) *httperror.H
stackCount = len(stacks)
}
containersStats, err := stats.CalculateContainerStats(r.Context(), cli, info.Swarm.ControlAvailable, containers)
if err != nil {
return httperror.InternalServerError("Unable to retrieve Docker containers stats", err)
}
resp = dashboardResponse{
Images: imagesCounters{
Total: len(images),
Size: totalSize,
},
Services: len(services),
Containers: containersStats,
Containers: docker.CalculateContainerStats(containers),
Networks: len(networks),
Volumes: len(volumes),
Stacks: stackCount,
@@ -163,5 +159,7 @@ func (h *Handler) dashboard(w http.ResponseWriter, r *http.Request) *httperror.H
return nil
})
return response.TxResponse(w, resp, err)
return errors.TxResponse(err, func() *httperror.HandlerError {
return response.JSON(w, resp)
})
}
@@ -11,7 +11,6 @@ import (
"github.com/portainer/portainer/api/internal/testhelpers"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestHandler_getDockerStacks(t *testing.T) {
@@ -70,7 +69,7 @@ func TestHandler_getDockerStacks(t *testing.T) {
stacksList, err := GetDockerStacks(datastore, &security.RestrictedRequestContext{
IsAdmin: true,
}, environment.ID, containers, services)
require.NoError(t, err)
assert.NoError(t, err)
assert.Len(t, stacksList, 3)
expectedStacks := []StackViewModel{
@@ -10,7 +10,6 @@ import (
"github.com/portainer/portainer/api/roar"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
)
type edgeGroupCreatePayload struct {
@@ -112,5 +111,5 @@ func (handler *Handler) edgeGroupCreate(w http.ResponseWriter, r *http.Request)
return nil
})
return response.TxResponse(w, shadowedEdgeGroup{EdgeGroup: *edgeGroup}, err)
return txResponse(w, shadowedEdgeGroup{EdgeGroup: *edgeGroup}, err)
}
@@ -32,8 +32,16 @@ func (handler *Handler) edgeGroupDelete(w http.ResponseWriter, r *http.Request)
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
return deleteEdgeGroup(tx, portainer.EdgeGroupID(edgeGroupID))
})
if err != nil {
var httpErr *httperror.HandlerError
if errors.As(err, &httpErr) {
return httpErr
}
return response.TxEmptyResponse(w, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.Empty(w)
}
func deleteEdgeGroup(tx dataservices.DataStoreTx, ID portainer.EdgeGroupID) error {
@@ -8,7 +8,6 @@ import (
"github.com/portainer/portainer/api/roar"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
)
// @id EdgeGroupInspect
@@ -37,7 +36,7 @@ func (handler *Handler) edgeGroupInspect(w http.ResponseWriter, r *http.Request)
edgeGroup.Endpoints = edgeGroup.EndpointIDs.ToSlice()
return response.TxResponse(w, shadowedEdgeGroup{EdgeGroup: *edgeGroup}, err)
return txResponse(w, shadowedEdgeGroup{EdgeGroup: *edgeGroup}, err)
}
func getEdgeGroup(tx dataservices.DataStoreTx, ID portainer.EdgeGroupID) (*portainer.EdgeGroup, error) {
@@ -105,7 +105,7 @@ func TestEmptyEdgeGroupInspectHandler(t *testing.T) {
// Make sure the frontend does not get a null value but a [] instead
require.NotNil(t, responseGroup.Endpoints)
require.Empty(t, responseGroup.Endpoints)
require.Len(t, responseGroup.Endpoints, 0)
}
func TestDynamicEdgeGroupInspectHandler(t *testing.T) {
@@ -9,7 +9,6 @@ import (
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/roar"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/response"
)
type shadowedEdgeGroup struct {
@@ -45,7 +44,7 @@ func (handler *Handler) edgeGroupList(w http.ResponseWriter, r *http.Request) *h
return err
})
return response.TxResponse(w, decoratedEdgeGroups, err)
return txResponse(w, decoratedEdgeGroups, err)
}
func getEdgeGroupList(tx dataservices.DataStoreTx) ([]decoratedEdgeGroup, error) {
@@ -47,7 +47,7 @@ func Test_getEndpointTypes(t *testing.T) {
for _, test := range tests {
ans, err := getEndpointTypes(datastore, roar.FromSlice(test.endpointIds))
require.NoError(t, err, "getEndpointTypes shouldn't fail")
assert.NoError(t, err, "getEndpointTypes shouldn't fail")
assert.ElementsMatch(t, test.expected, ans, "getEndpointTypes expected to return %b for %v, but returned %b", test.expected, test.endpointIds, ans)
}
@@ -57,7 +57,7 @@ func Test_getEndpointTypes_failWhenEndpointDontExist(t *testing.T) {
datastore := testhelpers.NewDatastore(testhelpers.WithEndpoints([]portainer.Endpoint{}))
_, err := getEndpointTypes(datastore, roar.FromSlice([]portainer.EndpointID{1}))
require.Error(t, err, "getEndpointTypes should fail")
assert.Error(t, err, "getEndpointTypes should fail")
}
func TestEdgeGroupListHandler(t *testing.T) {
@@ -112,5 +112,5 @@ func TestEdgeGroupListHandler(t *testing.T) {
require.Len(t, responseGroups, 1)
require.ElementsMatch(t, []portainer.EndpointID{1, 2, 3}, responseGroups[0].Endpoints)
require.Empty(t, responseGroups[0].TrustedEndpoints)
require.Len(t, responseGroups[0].TrustedEndpoints, 0)
}
@@ -13,7 +13,6 @@ import (
"github.com/portainer/portainer/api/slicesx"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
)
type edgeGroupUpdatePayload struct {
@@ -159,7 +158,7 @@ func (handler *Handler) edgeGroupUpdate(w http.ResponseWriter, r *http.Request)
return nil
})
return response.TxResponse(w, shadowedEdgeGroup{EdgeGroup: *edgeGroup}, err)
return txResponse(w, shadowedEdgeGroup{EdgeGroup: *edgeGroup}, err)
}
func (handler *Handler) updateEndpointStacks(tx dataservices.DataStoreTx, endpoint *portainer.Endpoint, edgeGroups []portainer.EdgeGroup, edgeStacks []portainer.EdgeStack) error {
+15
View File
@@ -1,12 +1,14 @@
package edgegroups
import (
"errors"
"net/http"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/http/security"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/response"
"github.com/gorilla/mux"
)
@@ -36,3 +38,16 @@ func NewHandler(bouncer security.BouncerService) *Handler {
return h
}
func txResponse(w http.ResponseWriter, r any, err error) *httperror.HandlerError {
if err != nil {
var handlerError *httperror.HandlerError
if errors.As(err, &handlerError) {
return handlerError
}
return httperror.InternalServerError("Unexpected error", err)
}
return response.JSON(w, r)
}
+8 -7
View File
@@ -15,7 +15,6 @@ import (
"github.com/portainer/portainer/api/internal/endpointutils"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
"github.com/portainer/portainer/pkg/validate"
)
@@ -86,18 +85,19 @@ func (payload *edgeJobCreateFromFileContentPayload) Validate(r *http.Request) er
// @router /edge_jobs/create/string [post]
func (handler *Handler) createEdgeJobFromFileContent(w http.ResponseWriter, r *http.Request) *httperror.HandlerError {
var payload edgeJobCreateFromFileContentPayload
if err := request.DecodeAndValidateJSONPayload(r, &payload); err != nil {
err := request.DecodeAndValidateJSONPayload(r, &payload)
if err != nil {
return httperror.BadRequest("Invalid request payload", err)
}
var edgeJob *portainer.EdgeJob
var err error
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
edgeJob, err = handler.createEdgeJob(tx, &payload.edgeJobBasePayload, []byte(payload.FileContent))
return err
})
return response.TxResponse(w, edgeJob, err)
return txResponse(w, edgeJob, err)
}
func (handler *Handler) createEdgeJob(tx dataservices.DataStoreTx, payload *edgeJobBasePayload, fileContent []byte) (*portainer.EdgeJob, error) {
@@ -191,18 +191,19 @@ func (payload *edgeJobCreateFromFilePayload) Validate(r *http.Request) error {
// @router /edge_jobs/create/file [post]
func (handler *Handler) createEdgeJobFromFile(w http.ResponseWriter, r *http.Request) *httperror.HandlerError {
payload := &edgeJobCreateFromFilePayload{}
if err := payload.Validate(r); err != nil {
err := payload.Validate(r)
if err != nil {
return httperror.BadRequest("Invalid request payload", err)
}
var edgeJob *portainer.EdgeJob
var err error
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
edgeJob, err = handler.createEdgeJob(tx, &payload.edgeJobBasePayload, payload.File)
return err
})
return response.TxResponse(w, edgeJob, err)
return txResponse(w, edgeJob, err)
}
func (handler *Handler) createEdgeJobObjectFromPayload(tx dataservices.DataStoreTx, payload *edgeJobBasePayload) *portainer.EdgeJob {
@@ -1,158 +0,0 @@
package edgejobs
import (
"bytes"
"encoding/json"
"io"
"mime/multipart"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/gorilla/mux"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/datastore"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
)
type mockFileService struct {
mock.Mock
portainer.FileService
}
func (m *mockFileService) StoreEdgeJobFileFromBytes(id string, file []byte) (string, error) {
args := m.Called(id, file)
return args.String(0), args.Error(1)
}
func (m *mockFileService) GetEdgeJobFolder(id string) string {
args := m.Called(id)
return args.String(0)
}
func (m *mockFileService) RemoveDirectory(path string) error {
args := m.Called(path)
return args.Error(0)
}
func initStore(t *testing.T) *datastore.Store {
_, store := datastore.MustNewTestStore(t, true, true)
require.NotNil(t, store)
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
require.NoError(t, tx.Endpoint().Create(&portainer.Endpoint{
ID: 1,
Name: "endpoint-1",
EdgeID: "edge-id-1",
GroupID: 1,
Type: portainer.EdgeAgentOnDockerEnvironment,
UserTrusted: true,
}))
require.NoError(t, tx.Endpoint().Create(&portainer.Endpoint{
ID: 2,
Name: "endpoint-2",
EdgeID: "edge-id-2",
GroupID: 1,
Type: portainer.EdgeAgentOnDockerEnvironment,
UserTrusted: false,
}))
return nil
}))
return store
}
func Test_edgeJobCreate_StringMethod_Success(t *testing.T) {
store := initStore(t)
fileService := &mockFileService{}
fileService.On("StoreEdgeJobFileFromBytes", mock.Anything, mock.Anything).Return("testfile.txt", nil)
handler := &Handler{
DataStore: store,
FileService: fileService,
}
payload := edgeJobCreateFromFileContentPayload{
edgeJobBasePayload: edgeJobBasePayload{
Name: "testjob",
CronExpression: "* * * * *",
Endpoints: []portainer.EndpointID{1, 2},
},
FileContent: "echo hello",
}
body, _ := json.Marshal(payload)
req := httptest.NewRequest(http.MethodPost, "/edge_jobs/create/string", bytes.NewReader(body))
req = mux.SetURLVars(req, map[string]string{"method": "string"})
w := httptest.NewRecorder()
// Call handler
errh := handler.edgeJobCreate(w, req)
require.Nil(t, errh)
require.Equal(t, http.StatusOK, w.Result().StatusCode)
// Get edge job ID from response
var resp struct {
ID int `json:"Id"`
}
require.NoError(t, json.NewDecoder(w.Body).Decode(&resp))
edgeJob, err := store.EdgeJob().Read(portainer.EdgeJobID(resp.ID))
require.NoError(t, err)
require.Len(t, edgeJob.Endpoints, 2)
require.Contains(t, edgeJob.Endpoints, portainer.EndpointID(1))
}
func Test_edgeJobCreate_FileMethod_Success(t *testing.T) {
store := initStore(t)
fileService := &mockFileService{}
fileService.On("StoreEdgeJobFileFromBytes", mock.Anything, mock.Anything).Return("testfile.txt", nil)
handler := &Handler{
DataStore: store,
FileService: fileService,
}
var body bytes.Buffer
writer := multipart.NewWriter(&body)
require.NoError(t, writer.WriteField("Name", "testjob"))
require.NoError(t, writer.WriteField("CronExpression", "* * * * *"))
require.NoError(t, writer.WriteField("Endpoints", "[1,2]"))
fileWriter, err := writer.CreateFormFile("file", "test.txt")
require.NoError(t, err)
_, err = io.Copy(fileWriter, strings.NewReader("echo hello"))
require.NoError(t, err)
require.NoError(t, writer.Close())
req := httptest.NewRequest(http.MethodPost, "/edge_jobs/create/file", &body)
req = mux.SetURLVars(req, map[string]string{"method": "file"})
req.Header.Set("Content-Type", writer.FormDataContentType())
w := httptest.NewRecorder()
handlerErr := handler.edgeJobCreate(w, req)
require.Nil(t, handlerErr)
require.Equal(t, http.StatusOK, w.Result().StatusCode)
var resp struct {
ID int `json:"Id"`
}
require.NoError(t, json.NewDecoder(w.Body).Decode(&resp))
edgeJob, err := store.EdgeJob().Read(portainer.EdgeJobID(resp.ID))
require.NoError(t, err)
require.Len(t, edgeJob.Endpoints, 2)
require.Contains(t, edgeJob.Endpoints, portainer.EndpointID(1))
}
+11 -3
View File
@@ -1,6 +1,7 @@
package edgejobs
import (
"errors"
"maps"
"net/http"
"strconv"
@@ -34,11 +35,18 @@ func (handler *Handler) edgeJobDelete(w http.ResponseWriter, r *http.Request) *h
return httperror.BadRequest("Invalid Edge job identifier route variable", err)
}
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
if err := handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
return handler.deleteEdgeJob(tx, portainer.EdgeJobID(edgeJobID))
})
}); err != nil {
var handlerError *httperror.HandlerError
if errors.As(err, &handlerError) {
return handlerError
}
return response.TxEmptyResponse(w, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.Empty(w)
}
func (handler *Handler) deleteEdgeJob(tx dataservices.DataStoreTx, edgeJobID portainer.EdgeJobID) error {
@@ -1,6 +1,7 @@
package edgejobs
import (
"errors"
"net/http"
"slices"
"strconv"
@@ -53,7 +54,7 @@ func (handler *Handler) edgeJobTasksClear(w http.ResponseWriter, r *http.Request
}
}
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
if err := handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
updateEdgeJobFn := func(edgeJob *portainer.EdgeJob, endpointID portainer.EndpointID, endpointsFromGroups []portainer.EndpointID) error {
mutationFn(edgeJob, endpointID, endpointsFromGroups)
@@ -61,9 +62,16 @@ func (handler *Handler) edgeJobTasksClear(w http.ResponseWriter, r *http.Request
}
return handler.clearEdgeJobTaskLogs(tx, portainer.EdgeJobID(edgeJobID), portainer.EndpointID(taskID), updateEdgeJobFn)
})
}); err != nil {
var handlerError *httperror.HandlerError
if errors.As(err, &handlerError) {
return handlerError
}
return response.TxEmptyResponse(w, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.Empty(w)
}
func (handler *Handler) clearEdgeJobTaskLogs(tx dataservices.DataStoreTx, edgeJobID portainer.EdgeJobID, endpointID portainer.EndpointID, updateEdgeJob func(*portainer.EdgeJob, portainer.EndpointID, []portainer.EndpointID) error) error {
@@ -1,6 +1,7 @@
package edgejobs
import (
"errors"
"net/http"
"slices"
@@ -38,7 +39,7 @@ func (handler *Handler) edgeJobTasksCollect(w http.ResponseWriter, r *http.Reque
return httperror.BadRequest("Invalid Task identifier route variable", err)
}
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
if err := handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
edgeJob, err := tx.EdgeJob().Read(portainer.EdgeJobID(edgeJobID))
if tx.IsErrObjectNotFound(err) {
return httperror.NotFound("Unable to find an Edge job with the specified identifier inside the database", err)
@@ -80,7 +81,14 @@ func (handler *Handler) edgeJobTasksCollect(w http.ResponseWriter, r *http.Reque
}
return nil
})
}); err != nil {
var handlerError *httperror.HandlerError
if errors.As(err, &handlerError) {
return handlerError
}
return response.TxEmptyResponse(w, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.Empty(w)
}
+22 -25
View File
@@ -13,7 +13,6 @@ import (
"github.com/portainer/portainer/api/internal/edge"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
)
type taskContainer struct {
@@ -50,33 +49,31 @@ func (handler *Handler) edgeJobTasksList(w http.ResponseWriter, r *http.Request)
return err
})
return response.TxFuncResponse(err, func() *httperror.HandlerError {
results := filters.SearchOrderAndPaginate(tasks, params, filters.Config[*taskContainer]{
SearchAccessors: []filters.SearchAccessor[*taskContainer]{
func(tc *taskContainer) (string, error) {
switch tc.LogsStatus {
case portainer.EdgeJobLogsStatusPending:
return "pending", nil
case 0, portainer.EdgeJobLogsStatusIdle:
return "idle", nil
case portainer.EdgeJobLogsStatusCollected:
return "collected", nil
}
return "", errors.New("unknown state")
},
func(tc *taskContainer) (string, error) {
return tc.EndpointName, nil
},
results := filters.SearchOrderAndPaginate(tasks, params, filters.Config[*taskContainer]{
SearchAccessors: []filters.SearchAccessor[*taskContainer]{
func(tc *taskContainer) (string, error) {
switch tc.LogsStatus {
case portainer.EdgeJobLogsStatusPending:
return "pending", nil
case 0, portainer.EdgeJobLogsStatusIdle:
return "idle", nil
case portainer.EdgeJobLogsStatusCollected:
return "collected", nil
}
return "", errors.New("unknown state")
},
SortBindings: []filters.SortBinding[*taskContainer]{
{Key: "EndpointName", Fn: func(a, b *taskContainer) int { return strings.Compare(a.EndpointName, b.EndpointName) }},
func(tc *taskContainer) (string, error) {
return tc.EndpointName, nil
},
})
filters.ApplyFilterResultsHeaders(&w, results)
return response.JSON(w, results.Items)
},
SortBindings: []filters.SortBinding[*taskContainer]{
{Key: "EndpointName", Fn: func(a, b *taskContainer) int { return strings.Compare(a.EndpointName, b.EndpointName) }},
},
})
filters.ApplyFilterResultsHeaders(&w, results)
return txResponse(w, results.Items, err)
}
func listEdgeJobTasks(tx dataservices.DataStoreTx, edgeJobID portainer.EdgeJobID) ([]*taskContainer, error) {
@@ -11,7 +11,6 @@ import (
"github.com/portainer/portainer/api/datastore"
"github.com/portainer/portainer/api/internal/testhelpers"
"github.com/portainer/portainer/api/roar"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -69,16 +68,14 @@ func Test_EdgeJobTasksListHandler(t *testing.T) {
tcStr := rr.Header().Get("x-total-count")
assert.NotEmpty(t, tcStr)
totalCount, err := strconv.Atoi(tcStr)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, expectedCount, totalCount)
taStr := rr.Header().Get("x-total-available")
assert.NotEmpty(t, taStr)
totalAvailable, err := strconv.Atoi(taStr)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, envCount, totalAvailable)
}
+1 -2
View File
@@ -14,7 +14,6 @@ import (
"github.com/portainer/portainer/api/internal/endpointutils"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
"github.com/portainer/portainer/pkg/validate"
)
@@ -67,7 +66,7 @@ func (handler *Handler) edgeJobUpdate(w http.ResponseWriter, r *http.Request) *h
return err
})
return response.TxResponse(w, edgeJob, err)
return txResponse(w, edgeJob, err)
}
func (handler *Handler) updateEdgeJob(tx dataservices.DataStoreTx, edgeJobID portainer.EdgeJobID, payload edgeJobUpdatePayload) (*portainer.EdgeJob, error) {
+15
View File
@@ -1,12 +1,14 @@
package edgejobs
import (
"errors"
"net/http"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/http/security"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/response"
"github.com/gorilla/mux"
)
@@ -58,3 +60,16 @@ func convertEndpointsToMetaObject(endpoints []portainer.EndpointID) map[portaine
return endpointsMap
}
func txResponse(w http.ResponseWriter, r any, err error) *httperror.HandlerError {
if err != nil {
var handlerError *httperror.HandlerError
if errors.As(err, &handlerError) {
return handlerError
}
return httperror.InternalServerError("Unexpected error", err)
}
return response.JSON(w, r)
}
@@ -1,6 +1,7 @@
package edgestacks
import (
"errors"
"net/http"
"strconv"
@@ -29,11 +30,18 @@ func (handler *Handler) edgeStackDelete(w http.ResponseWriter, r *http.Request)
return httperror.BadRequest("Invalid edge stack identifier route variable", err)
}
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
if err := handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
return handler.deleteEdgeStack(tx, portainer.EdgeStackID(edgeStackID))
})
}); err != nil {
var httpErr *httperror.HandlerError
if errors.As(err, &httpErr) {
return httpErr
}
return response.TxEmptyResponse(w, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.Empty(w)
}
func (handler *Handler) deleteEdgeStack(tx dataservices.DataStoreTx, edgeStackID portainer.EdgeStackID) error {
@@ -96,7 +96,12 @@ func (handler *Handler) edgeStackStatusUpdate(w http.ResponseWriter, r *http.Req
return nil
}); err != nil {
return response.TxErrorResponse(err)
var httpErr *httperror.HandlerError
if errors.As(err, &httpErr) {
return httpErr
}
return httperror.InternalServerError("Unexpected error", err)
}
if ok, _ := strconv.ParseBool(r.Header.Get("X-Portainer-No-Body")); ok {
@@ -66,7 +66,12 @@ func (handler *Handler) edgeStackUpdate(w http.ResponseWriter, r *http.Request)
stack, err = handler.updateEdgeStack(tx, portainer.EdgeStackID(stackID), payload)
return err
}); err != nil {
return response.TxErrorResponse(err)
var httpErr *httperror.HandlerError
if errors.As(err, &httpErr) {
return httpErr
}
return httperror.InternalServerError("Unexpected error", err)
}
if err := fillEdgeStackStatus(handler.DataStore, stack); err != nil {
@@ -5,9 +5,7 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/internal/testhelpers"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_hasKubeEndpoint(t *testing.T) {
@@ -42,7 +40,7 @@ func Test_hasKubeEndpoint(t *testing.T) {
for _, test := range tests {
ans, err := hasKubeEndpoint(datastore.Endpoint(), test.endpointIds)
require.NoError(t, err, "hasKubeEndpoint shouldn't fail")
assert.NoError(t, err, "hasKubeEndpoint shouldn't fail")
assert.Equal(t, test.expected, ans, "hasKubeEndpoint expected to return %b for %v, but returned %b", test.expected, test.endpointIds, ans)
}
@@ -52,7 +50,7 @@ func Test_hasKubeEndpoint_failWhenEndpointDontExist(t *testing.T) {
datastore := testhelpers.NewDatastore(testhelpers.WithEndpoints([]portainer.Endpoint{}))
_, err := hasKubeEndpoint(datastore.Endpoint(), []portainer.EndpointID{1})
require.Error(t, err, "hasKubeEndpoint should fail")
assert.Error(t, err, "hasKubeEndpoint should fail")
}
func Test_hasDockerEndpoint(t *testing.T) {
@@ -87,7 +85,7 @@ func Test_hasDockerEndpoint(t *testing.T) {
for _, test := range tests {
ans, err := hasDockerEndpoint(datastore.Endpoint(), test.endpointIds)
require.NoError(t, err, "hasDockerEndpoint shouldn't fail")
assert.NoError(t, err, "hasDockerEndpoint shouldn't fail")
assert.Equal(t, test.expected, ans, "hasDockerEndpoint expected to return %b for %v, but returned %b", test.expected, test.endpointIds, ans)
}
@@ -97,5 +95,5 @@ func Test_hasDockerEndpoint_failWhenEndpointDontExist(t *testing.T) {
datastore := testhelpers.NewDatastore(testhelpers.WithEndpoints([]portainer.Endpoint{}))
_, err := hasDockerEndpoint(datastore.Endpoint(), []portainer.EndpointID{1})
require.Error(t, err, "hasDockerEndpoint should fail")
assert.Error(t, err, "hasDockerEndpoint should fail")
}
@@ -49,18 +49,26 @@ func (payload *endpointGroupCreatePayload) Validate(r *http.Request) error {
// @router /endpoint_groups [post]
func (handler *Handler) endpointGroupCreate(w http.ResponseWriter, r *http.Request) *httperror.HandlerError {
var payload endpointGroupCreatePayload
if err := request.DecodeAndValidateJSONPayload(r, &payload); err != nil {
err := request.DecodeAndValidateJSONPayload(r, &payload)
if err != nil {
return httperror.BadRequest("Invalid request payload", err)
}
var endpointGroup *portainer.EndpointGroup
var err error
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
endpointGroup, err = handler.createEndpointGroup(tx, payload)
return err
})
if err != nil {
var httpErr *httperror.HandlerError
if errors.As(err, &httpErr) {
return httpErr
}
return response.TxResponse(w, endpointGroup, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.JSON(w, endpointGroup)
}
func (handler *Handler) createEndpointGroup(tx dataservices.DataStoreTx, payload endpointGroupCreatePayload) (*portainer.EndpointGroup, error) {
@@ -37,8 +37,16 @@ func (handler *Handler) endpointGroupDelete(w http.ResponseWriter, r *http.Reque
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
return handler.deleteEndpointGroup(tx, portainer.EndpointGroupID(endpointGroupID))
})
if err != nil {
var httpErr *httperror.HandlerError
if errors.As(err, &httpErr) {
return httpErr
}
return response.TxEmptyResponse(w, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.Empty(w)
}
func (handler *Handler) deleteEndpointGroup(tx dataservices.DataStoreTx, endpointGroupID portainer.EndpointGroupID) error {
@@ -1,6 +1,7 @@
package endpointgroups
import (
"errors"
"net/http"
portainer "github.com/portainer/portainer/api"
@@ -38,8 +39,16 @@ func (handler *Handler) endpointGroupAddEndpoint(w http.ResponseWriter, r *http.
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
return handler.addEndpoint(tx, portainer.EndpointGroupID(endpointGroupID), portainer.EndpointID(endpointID))
})
if err != nil {
var httpErr *httperror.HandlerError
if errors.As(err, &httpErr) {
return httpErr
}
return response.TxEmptyResponse(w, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.Empty(w)
}
func (handler *Handler) addEndpoint(tx dataservices.DataStoreTx, endpointGroupID portainer.EndpointGroupID, endpointID portainer.EndpointID) error {
@@ -1,6 +1,7 @@
package endpointgroups
import (
"errors"
"net/http"
portainer "github.com/portainer/portainer/api"
@@ -37,8 +38,16 @@ func (handler *Handler) endpointGroupDeleteEndpoint(w http.ResponseWriter, r *ht
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
return handler.removeEndpoint(tx, portainer.EndpointGroupID(endpointGroupID), portainer.EndpointID(endpointID))
})
if err != nil {
var httpErr *httperror.HandlerError
if errors.As(err, &httpErr) {
return httpErr
}
return response.TxEmptyResponse(w, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.Empty(w)
}
func (handler *Handler) removeEndpoint(tx dataservices.DataStoreTx, endpointGroupID portainer.EndpointGroupID, endpointID portainer.EndpointID) error {
@@ -1,6 +1,7 @@
package endpointgroups
import (
"errors"
"net/http"
"reflect"
@@ -60,12 +61,20 @@ func (handler *Handler) endpointGroupUpdate(w http.ResponseWriter, r *http.Reque
var endpointGroup *portainer.EndpointGroup
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
if err := handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
endpointGroup, err = handler.updateEndpointGroup(tx, portainer.EndpointGroupID(endpointGroupID), payload)
return err
})
return response.TxResponse(w, endpointGroup, err)
return err
}); err != nil {
var httpErr *httperror.HandlerError
if errors.As(err, &httpErr) {
return httpErr
}
return httperror.InternalServerError("Unexpected error", err)
}
return response.JSON(w, endpointGroup)
}
func (handler *Handler) updateEndpointGroup(tx dataservices.DataStoreTx, endpointGroupID portainer.EndpointGroupID, payload endpointGroupUpdatePayload) (*portainer.EndpointGroup, error) {
+10 -3
View File
@@ -62,11 +62,18 @@ func (handler *Handler) endpointDelete(w http.ResponseWriter, r *http.Request) *
return httperror.BadRequest("Invalid boolean query parameter", err)
}
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
if err := handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
return handler.deleteEndpoint(tx, portainer.EndpointID(endpointID), deleteCluster)
})
}); err != nil {
var handlerError *httperror.HandlerError
if errors.As(err, &handlerError) {
return handlerError
}
return response.TxEmptyResponse(w, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.Empty(w)
}
// @id EndpointDeleteBatch
@@ -15,7 +15,6 @@ import (
"github.com/segmentio/encoding/json"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type endpointListTest struct {
@@ -87,9 +86,9 @@ func Test_EndpointList_AgentVersion(t *testing.T) {
req := buildEndpointListRequest(query)
resp, err := doEndpointListRequest(req, handler, is)
require.NoError(t, err)
is.NoError(err)
is.Len(resp, len(test.expected))
is.Equal(len(test.expected), len(resp))
respIds := []portainer.EndpointID{}
@@ -165,9 +164,9 @@ func Test_endpointList_edgeFilter(t *testing.T) {
req := buildEndpointListRequest(query)
resp, err := doEndpointListRequest(req, handler, is)
require.NoError(t, err)
is.NoError(err)
is.Len(resp, len(test.expected))
is.Equal(len(test.expected), len(resp))
respIds := []portainer.EndpointID{}
@@ -181,15 +180,16 @@ func Test_endpointList_edgeFilter(t *testing.T) {
}
func setupEndpointListHandler(t *testing.T, endpoints []portainer.Endpoint) *Handler {
is := assert.New(t)
_, store := datastore.MustNewTestStore(t, true, true)
for _, endpoint := range endpoints {
err := store.Endpoint().Create(&endpoint)
require.NoError(t, err, "error creating environment")
is.NoError(err, "error creating environment")
}
err := store.User().Create(&portainer.User{Username: "admin", Role: portainer.AdministratorRole})
require.NoError(t, err, "error creating a user")
is.NoError(err, "error creating a user")
bouncer := testhelpers.NewTestRequestBouncer()
@@ -35,12 +35,19 @@ func (handler *Handler) endpointRegistriesList(w http.ResponseWriter, r *http.Re
}
var registries []portainer.Registry
err = handler.DataStore.ViewTx(func(tx dataservices.DataStoreTx) error {
if err := handler.DataStore.ViewTx(func(tx dataservices.DataStoreTx) error {
registries, err = handler.listRegistries(tx, r, portainer.EndpointID(endpointID))
return err
})
}); err != nil {
var httpErr *httperror.HandlerError
if errors.As(err, &httpErr) {
return httpErr
}
return response.TxResponse(w, registries, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.JSON(w, registries)
}
func (handler *Handler) listRegistries(tx dataservices.DataStoreTx, r *http.Request, endpointID portainer.EndpointID) ([]portainer.Registry, error) {
@@ -1,6 +1,7 @@
package endpoints
import (
"errors"
"net/http"
portainer "github.com/portainer/portainer/api"
@@ -52,8 +53,16 @@ func (handler *Handler) endpointRegistryAccess(w http.ResponseWriter, r *http.Re
err = handler.DataStore.UpdateTx(func(tx dataservices.DataStoreTx) error {
return handler.updateRegistryAccess(tx, r, portainer.EndpointID(endpointID), portainer.RegistryID(registryID))
})
if err != nil {
var httpErr *httperror.HandlerError
if errors.As(err, &httpErr) {
return httpErr
}
return response.TxEmptyResponse(w, err)
return httperror.InternalServerError("Unexpected error", err)
}
return response.Empty(w)
}
func (handler *Handler) updateRegistryAccess(tx dataservices.DataStoreTx, r *http.Request, endpointID portainer.EndpointID, registryID portainer.RegistryID) error {
+44 -37
View File
@@ -256,7 +256,7 @@ func (handler *Handler) filterEndpointsByQuery(
return filteredEndpoints, totalAvailableEndpoints, nil
}
func endpointStatusInStackMatchesFilter(stackStatus *portainer.EdgeStackStatusForEnv, envId portainer.EndpointID, statusFilter portainer.EdgeStackStatusType) bool {
func endpointStatusInStackMatchesFilter(stackStatus *portainer.EdgeStackStatusForEnv, statusFilter portainer.EdgeStackStatusType) bool {
// consider that if the env has no status in the stack it is in Pending state
if statusFilter == portainer.EdgeStackStatusPending {
return stackStatus == nil || len(stackStatus.Status) == 0
@@ -272,55 +272,62 @@ func endpointStatusInStackMatchesFilter(stackStatus *portainer.EdgeStackStatusFo
}
func filterEndpointsByEdgeStack(endpoints []portainer.Endpoint, edgeStackId portainer.EdgeStackID, statusFilter *portainer.EdgeStackStatusType, datastore dataservices.DataStore) ([]portainer.Endpoint, error) {
stack, err := datastore.EdgeStack().EdgeStack(edgeStackId)
if err != nil {
return nil, errors.WithMessage(err, "Unable to retrieve edge stack from the database")
}
envIds := roar.Roar[portainer.EndpointID]{}
for _, edgeGroupdId := range stack.EdgeGroups {
edgeGroup, err := datastore.EdgeGroup().Read(edgeGroupdId)
var filteredEndpoints []portainer.Endpoint
if err := datastore.ViewTx(func(tx dataservices.DataStoreTx) error {
stack, err := tx.EdgeStack().EdgeStack(edgeStackId)
if err != nil {
return nil, errors.WithMessage(err, "Unable to retrieve edge group from the database")
return errors.WithMessage(err, "Unable to retrieve edge stack from the database")
}
if edgeGroup.Dynamic {
endpointIDs, err := edgegroups.GetEndpointsByTags(datastore, edgeGroup.TagIDs, edgeGroup.PartialMatch)
envIds := roar.Roar[portainer.EndpointID]{}
for _, edgeGroupId := range stack.EdgeGroups {
edgeGroup, err := tx.EdgeGroup().Read(edgeGroupId)
if err != nil {
return nil, errors.WithMessage(err, "Unable to retrieve environments and environment groups for Edge group")
return errors.WithMessage(err, "Unable to retrieve edge group from the database")
}
edgeGroup.EndpointIDs = roar.FromSlice(endpointIDs)
if edgeGroup.Dynamic {
endpointIDs, err := edgegroups.GetEndpointsByTags(tx, edgeGroup.TagIDs, edgeGroup.PartialMatch)
if err != nil {
return errors.WithMessage(err, "Unable to retrieve environments and environment groups for Edge group")
}
edgeGroup.EndpointIDs = roar.FromSlice(endpointIDs)
}
envIds.Union(edgeGroup.EndpointIDs)
}
envIds.Union(edgeGroup.EndpointIDs)
}
filteredEnvIds := roar.Roar[portainer.EndpointID]{}
filteredEnvIds.Union(envIds)
if statusFilter != nil {
var innerErr error
if statusFilter != nil {
var innerErr error
envIds.Iterate(func(envId portainer.EndpointID) bool {
edgeStackStatus, err := tx.EdgeStackStatus().Read(edgeStackId, envId)
if err != nil && !dataservices.IsErrObjectNotFound(err) {
innerErr = errors.WithMessagef(err, "Unable to retrieve edge stack status for environment %d", envId)
return false
}
if !endpointStatusInStackMatchesFilter(edgeStackStatus, *statusFilter) {
filteredEnvIds.Remove(envId)
}
envIds.Iterate(func(envId portainer.EndpointID) bool {
edgeStackStatus, err := datastore.EdgeStackStatus().Read(edgeStackId, envId)
if dataservices.IsErrObjectNotFound(err) {
return true
} else if err != nil {
innerErr = errors.WithMessagef(err, "Unable to retrieve edge stack status for environment %d", envId)
return false
})
if innerErr != nil {
return innerErr
}
if !endpointStatusInStackMatchesFilter(edgeStackStatus, portainer.EndpointID(envId), *statusFilter) {
envIds.Remove(envId)
}
return true
})
if innerErr != nil {
return nil, innerErr
}
filteredEndpoints = filteredEndpointsByIds(endpoints, filteredEnvIds)
return nil
}); err != nil {
return nil, err
}
filteredEndpoints := filteredEndpointsByIds(endpoints, envIds)
return filteredEndpoints, nil
}
+104 -35
View File
@@ -5,6 +5,7 @@ import (
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/datastore"
"github.com/portainer/portainer/api/http/security"
"github.com/portainer/portainer/api/internal/testhelpers"
@@ -189,7 +190,9 @@ func BenchmarkFilterEndpointsBySearchCriteria_PartialMatch(b *testing.B) {
searchString := "edge-group"
for b.Loop() {
b.ResetTimer()
for range b.N {
e := filterEndpointsBySearchCriteria(endpoints, endpointGroups, edgeGroups, tagsMap, searchString)
if len(e) != n {
b.FailNow()
@@ -235,9 +238,13 @@ func BenchmarkFilterEndpointsBySearchCriteria_FullMatch(b *testing.B) {
searchString := "edge-group"
for b.Loop() {
b.ResetTimer()
for range b.N {
e := filterEndpointsBySearchCriteria(endpoints, endpointGroups, edgeGroups, tagsMap, searchString)
require.Len(b, e, n)
if len(e) != n {
b.FailNow()
}
}
}
@@ -261,9 +268,9 @@ func runTest(t *testing.T, test filterTest, handler *Handler, endpoints []portai
&security.RestrictedRequestContext{IsAdmin: true},
)
require.NoError(t, err)
is.NoError(err)
is.Len(filteredEndpoints, len(test.expected))
is.Equal(len(test.expected), len(filteredEndpoints))
respIds := []portainer.EndpointID{}
@@ -275,15 +282,16 @@ func runTest(t *testing.T, test filterTest, handler *Handler, endpoints []portai
}
func setupFilterTest(t *testing.T, endpoints []portainer.Endpoint) *Handler {
is := assert.New(t)
_, store := datastore.MustNewTestStore(t, true, true)
for _, endpoint := range endpoints {
err := store.Endpoint().Create(&endpoint)
require.NoError(t, err, "error creating environment")
is.NoError(err, "error creating environment")
}
err := store.User().Create(&portainer.User{Username: "admin", Role: portainer.AdministratorRole})
require.NoError(t, err, "error creating a user")
is.NoError(err, "error creating a user")
bouncer := testhelpers.NewTestRequestBouncer()
handler := NewHandler(bouncer)
@@ -297,42 +305,103 @@ func TestFilterEndpointsByEdgeStack(t *testing.T) {
_, store := datastore.MustNewTestStore(t, false, false)
endpoints := []portainer.Endpoint{
{ID: 1, Name: "Endpoint 1"},
{ID: 2, Name: "Endpoint 2"},
{ID: 3, Name: "Endpoint 3"},
{ID: 1, Name: "Endpoint 1", Type: portainer.EdgeAgentOnDockerEnvironment, UserTrusted: true},
{ID: 2, Name: "Endpoint 2", TagIDs: []portainer.TagID{1}, Type: portainer.EdgeAgentOnDockerEnvironment, UserTrusted: true},
{ID: 3, Name: "Endpoint 3", TagIDs: []portainer.TagID{1}, Type: portainer.EdgeAgentOnDockerEnvironment, UserTrusted: true},
{ID: 4, Name: "Endpoint 4"},
}
edgeStackId := portainer.EdgeStackID(1)
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
require.NoError(t, tx.Tag().Create(&portainer.Tag{ID: 1, Name: "tag", Endpoints: map[portainer.EndpointID]bool{2: true, 3: true}}))
err := store.EdgeStack().Create(edgeStackId, &portainer.EdgeStack{
ID: edgeStackId,
Name: "Test Edge Stack",
EdgeGroups: []portainer.EdgeGroupID{1, 2},
for i := range endpoints {
require.NoError(t, tx.Endpoint().Create(&endpoints[i]))
}
require.NoError(t, tx.EdgeStack().Create(edgeStackId, &portainer.EdgeStack{
ID: edgeStackId,
Name: "Test Edge Stack",
EdgeGroups: []portainer.EdgeGroupID{1, 2},
}))
require.NoError(t, tx.EdgeGroup().Create(&portainer.EdgeGroup{
ID: 1,
Name: "Edge Group 1",
EndpointIDs: roar.FromSlice([]portainer.EndpointID{1}),
}))
require.NoError(t, tx.EdgeGroup().Create(&portainer.EdgeGroup{
ID: 2,
Name: "Edge Group 2",
Dynamic: true,
TagIDs: []portainer.TagID{1},
}))
require.NoError(t, tx.EdgeStackStatus().Create(edgeStackId, endpoints[0].ID, &portainer.EdgeStackStatusForEnv{
Status: []portainer.EdgeStackDeploymentStatus{{Type: portainer.EdgeStackStatusAcknowledged}}}))
return nil
}))
test := func(status *portainer.EdgeStackStatusType, expected []portainer.Endpoint) {
tmp := make([]portainer.Endpoint, len(endpoints))
require.Equal(t, 4, copy(tmp, endpoints))
es, err := filterEndpointsByEdgeStack(tmp, edgeStackId, status, store)
require.NoError(t, err)
// validate that the len is the same
require.Len(t, es, len(expected))
// and that all items are the expected ones
for i := range expected {
require.Contains(t, es, expected[i])
}
}
test(nil, []portainer.Endpoint{endpoints[0], endpoints[1], endpoints[2]})
status := portainer.EdgeStackStatusPending
test(&status, []portainer.Endpoint{endpoints[1], endpoints[2]})
status = portainer.EdgeStackStatusCompleted
test(&status, []portainer.Endpoint{})
status = portainer.EdgeStackStatusAcknowledged
test(&status, []portainer.Endpoint{endpoints[0]}) // that's the only one with an edge stack status in DB
}
func TestErrorsFilterEndpointsByEdgeStack(t *testing.T) {
t.Run("must error by edge stack not found", func(t *testing.T) {
_, store := datastore.MustNewTestStore(t, false, false)
require.NotNil(t, store)
_, err := filterEndpointsByEdgeStack([]portainer.Endpoint{}, 1, nil, store)
require.Error(t, err)
})
require.NoError(t, err)
err = store.EdgeGroup().Create(&portainer.EdgeGroup{
ID: 1,
Name: "Edge Group 1",
EndpointIDs: roar.FromSlice([]portainer.EndpointID{1}),
t.Run("must error by edge group not found", func(t *testing.T) {
_, store := datastore.MustNewTestStore(t, false, false)
require.NotNil(t, store)
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
require.NoError(t, tx.EdgeStack().Create(1, &portainer.EdgeStack{ID: 1, Name: "1", EdgeGroups: []portainer.EdgeGroupID{1}}))
return nil
}))
_, err := filterEndpointsByEdgeStack([]portainer.Endpoint{}, 1, nil, store)
require.Error(t, err)
})
require.NoError(t, err)
err = store.EdgeGroup().Create(&portainer.EdgeGroup{
ID: 2,
Name: "Edge Group 2",
EndpointIDs: roar.FromSlice([]portainer.EndpointID{2, 3}),
t.Run("must error by env tag not found", func(t *testing.T) {
_, store := datastore.MustNewTestStore(t, false, false)
require.NotNil(t, store)
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
require.NoError(t, tx.EdgeStack().Create(1, &portainer.EdgeStack{ID: 1, Name: "1", EdgeGroups: []portainer.EdgeGroupID{1}}))
require.NoError(t, tx.EdgeGroup().Create(&portainer.EdgeGroup{ID: 1, Name: "edge group", Dynamic: true, TagIDs: []portainer.TagID{1}}))
return nil
}))
_, err := filterEndpointsByEdgeStack([]portainer.Endpoint{}, 1, nil, store)
require.Error(t, err)
})
require.NoError(t, err)
es, err := filterEndpointsByEdgeStack(endpoints, edgeStackId, nil, store)
require.NoError(t, err)
require.Len(t, es, 3)
require.Contains(t, es, endpoints[0]) // Endpoint 1
require.Contains(t, es, endpoints[1]) // Endpoint 2
require.Contains(t, es, endpoints[2]) // Endpoint 3
require.NotContains(t, es, endpoints[3]) // Endpoint 4
}
func TestFilterEndpointsByEdgeGroup(t *testing.T) {
@@ -407,11 +476,11 @@ func TestFilterEndpointsByExcludeEdgeGroupIDs(t *testing.T) {
require.NoError(t, err)
require.Len(t, es, 3)
require.Equal(t, []portainer.Endpoint{
require.Equal(t, es, []portainer.Endpoint{
{ID: 2, Name: "Endpoint 2"},
{ID: 3, Name: "Endpoint 3"},
{ID: 4, Name: "Endpoint 4"},
}, es)
})
require.Len(t, egs, 1)
require.Equal(t, egs[0].ID, portainer.EdgeGroupID(2))
@@ -8,7 +8,6 @@ import (
"github.com/portainer/portainer/api/datastore"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_updateEdgeGroups(t *testing.T) {
@@ -34,7 +33,7 @@ func Test_updateEdgeGroups(t *testing.T) {
checkGroups := func(store *datastore.Store, is *assert.Assertions, groupIDs []portainer.EdgeGroupID, endpointID portainer.EndpointID) {
for _, groupID := range groupIDs {
group, err := store.EdgeGroup().Read(groupID)
require.NoError(t, err)
is.NoError(err)
is.True(group.EndpointIDs.Contains(endpointID),
"expected endpoint to be in group")
@@ -70,17 +69,17 @@ func Test_updateEdgeGroups(t *testing.T) {
_, store := datastore.MustNewTestStore(t, true, true)
err := store.Endpoint().Create(testCase.endpoint)
require.NoError(t, err)
is.NoError(err)
groups, err := createGroups(store, testCase.groupNames)
require.NoError(t, err)
is.NoError(err)
endpointGroups := groupsByName(groups, testCase.endpointGroupNames)
for _, group := range endpointGroups {
group.EndpointIDs.Add(testCase.endpoint.ID)
err = store.EdgeGroup().Update(group.ID, &group)
require.NoError(t, err)
is.NoError(err)
}
expectedGroups := groupsByName(groups, testCase.groupsToApply)
@@ -92,14 +91,14 @@ func Test_updateEdgeGroups(t *testing.T) {
err = store.UpdateTx(func(tx dataservices.DataStoreTx) error {
updated, err := updateEnvironmentEdgeGroups(tx, expectedIDs, testCase.endpoint.ID)
require.NoError(t, err)
is.NoError(err)
is.Equal(testCase.shouldNotBeUpdated, !updated)
return nil
})
require.NoError(t, err)
is.NoError(err)
checkGroups(store, is, expectedIDs, testCase.endpoint.ID)
}
@@ -6,9 +6,7 @@ import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/datastore"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_updateTags(t *testing.T) {
@@ -35,7 +33,7 @@ func Test_updateTags(t *testing.T) {
checkTags := func(store *datastore.Store, is *assert.Assertions, tagIDs []portainer.TagID, endpointID portainer.EndpointID) {
for _, tagID := range tagIDs {
tag, err := store.Tag().Read(tagID)
require.NoError(t, err)
is.NoError(err)
_, ok := tag.Endpoints[endpointID]
is.True(ok, "expected endpoint to be tagged")
@@ -79,23 +77,23 @@ func Test_updateTags(t *testing.T) {
_, store := datastore.MustNewTestStore(t, true, true)
err := store.Endpoint().Create(testCase.endpoint)
require.NoError(t, err)
is.NoError(err)
tags, err := createTags(store, testCase.tagNames)
require.NoError(t, err)
is.NoError(err)
endpointTags := tagsByName(tags, testCase.endpointTagNames)
for _, tag := range endpointTags {
tag.Endpoints[testCase.endpoint.ID] = true
err = store.Tag().Update(tag.ID, &tag)
require.NoError(t, err)
is.NoError(err)
}
endpointTagIDs := getIDs(endpointTags)
testCase.endpoint.TagIDs = endpointTagIDs
err = store.Endpoint().UpdateEndpoint(testCase.endpoint.ID, testCase.endpoint)
require.NoError(t, err)
is.NoError(err)
expectedTags := tagsByName(tags, testCase.tagsToApply)
expectedTagIDs := make([]portainer.TagID, len(expectedTags))
@@ -105,14 +103,14 @@ func Test_updateTags(t *testing.T) {
err = store.UpdateTx(func(tx dataservices.DataStoreTx) error {
updated, err := updateEnvironmentTags(tx, expectedTagIDs, testCase.endpoint.TagIDs, testCase.endpoint.ID)
require.NoError(t, err)
is.NoError(err)
is.Equal(testCase.shouldNotBeUpdated, !updated)
return nil
})
require.NoError(t, err)
is.NoError(err)
checkTags(store, is, expectedTagIDs, testCase.endpoint.ID)
}
+1 -1
View File
@@ -81,7 +81,7 @@ type Handler struct {
}
// @title PortainerCE API
// @version 2.34.0
// @version 2.33.3
// @description.markdown api-description.md
// @termsOfService
+3 -4
View File
@@ -16,7 +16,6 @@ import (
"github.com/portainer/portainer/pkg/libhelm/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_helmDelete(t *testing.T) {
@@ -25,13 +24,13 @@ func Test_helmDelete(t *testing.T) {
_, store := datastore.MustNewTestStore(t, true, true)
err := store.Endpoint().Create(&portainer.Endpoint{ID: 1})
require.NoError(t, err, "Error creating environment")
is.NoError(err, "Error creating environment")
err = store.User().Create(&portainer.User{Username: "admin", Role: portainer.AdministratorRole})
require.NoError(t, err, "Error creating a user")
is.NoError(err, "Error creating a user")
jwtService, err := jwt.NewService("1h", store)
require.NoError(t, err, "Error initiating jwt service")
is.NoError(err, "Error initiating jwt service")
kubernetesDeployer := exectest.NewKubernetesDeployer()
helmPackageManager := test.NewMockHelmPackageManager()
+4 -5
View File
@@ -19,7 +19,6 @@ import (
"github.com/segmentio/encoding/json"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_helmGet(t *testing.T) {
@@ -28,13 +27,13 @@ func Test_helmGet(t *testing.T) {
_, store := datastore.MustNewTestStore(t, true, true)
err := store.Endpoint().Create(&portainer.Endpoint{ID: 1})
require.NoError(t, err, "Error creating environment")
is.NoError(err, "Error creating environment")
err = store.User().Create(&portainer.User{Username: "admin", Role: portainer.AdministratorRole})
require.NoError(t, err, "Error creating a user")
is.NoError(err, "Error creating a user")
jwtService, err := jwt.NewService("1h", store)
require.NoError(t, err, "Error initiating jwt service")
is.NoError(err, "Error initiating jwt service")
kubernetesDeployer := exectest.NewKubernetesDeployer()
helmPackageManager := test.NewMockHelmPackageManager()
@@ -58,7 +57,7 @@ func Test_helmGet(t *testing.T) {
data := release.Release{}
body, err := io.ReadAll(rr.Body)
require.NoError(t, err, "ReadAll should not return error")
is.NoError(err, "ReadAll should not return error")
json.Unmarshal(body, &data)
is.Equal(http.StatusOK, rr.Code, "Status should be 200")
is.Equal("nginx-1", data.Name)
+5 -6
View File
@@ -19,7 +19,6 @@ import (
"github.com/segmentio/encoding/json"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_helmGetHistory(t *testing.T) {
@@ -28,13 +27,13 @@ func Test_helmGetHistory(t *testing.T) {
_, store := datastore.MustNewTestStore(t, true, true)
err := store.Endpoint().Create(&portainer.Endpoint{ID: 1})
require.NoError(t, err, "Error creating environment")
is.NoError(err, "Error creating environment")
err = store.User().Create(&portainer.User{Username: "admin", Role: portainer.AdministratorRole})
require.NoError(t, err, "Error creating a user")
is.NoError(err, "Error creating a user")
jwtService, err := jwt.NewService("1h", store)
require.NoError(t, err, "Error initiating jwt service")
is.NoError(err, "Error initiating jwt service")
kubernetesDeployer := exectest.NewKubernetesDeployer()
helmPackageManager := test.NewMockHelmPackageManager()
@@ -58,10 +57,10 @@ func Test_helmGetHistory(t *testing.T) {
data := []release.Release{}
body, err := io.ReadAll(rr.Body)
require.NoError(t, err, "ReadAll should not return error")
is.NoError(err, "ReadAll should not return error")
json.Unmarshal(body, &data)
is.Equal(http.StatusOK, rr.Code, "Status should be 200")
is.Len(data, 1)
is.Equal(1, len(data))
is.Equal("nginx-1", data[0].Name)
})
}
+8 -9
View File
@@ -20,7 +20,6 @@ import (
"github.com/segmentio/encoding/json"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_helmInstall(t *testing.T) {
@@ -29,13 +28,13 @@ func Test_helmInstall(t *testing.T) {
_, store := datastore.MustNewTestStore(t, true, true)
err := store.Endpoint().Create(&portainer.Endpoint{ID: 1})
require.NoError(t, err, "error creating environment")
is.NoError(err, "error creating environment")
err = store.User().Create(&portainer.User{Username: "admin", Role: portainer.AdministratorRole})
require.NoError(t, err, "error creating a user")
is.NoError(err, "error creating a user")
jwtService, err := jwt.NewService("1h", store)
require.NoError(t, err, "Error initiating jwt service")
is.NoError(err, "Error initiating jwt service")
kubernetesDeployer := exectest.NewKubernetesDeployer()
helmPackageManager := test.NewMockHelmPackageManager()
@@ -47,7 +46,7 @@ func Test_helmInstall(t *testing.T) {
// Install a single chart. We expect to get these values back
options := options.InstallOptions{Name: "nginx-1", Chart: "nginx", Namespace: "default", Repo: "https://charts.bitnami.com/bitnami"}
optdata, err := json.Marshal(options)
require.NoError(t, err)
is.NoError(err)
t.Run("helmInstall succeeds with admin user", func(t *testing.T) {
req := httptest.NewRequest(http.MethodPost, "/1/kubernetes/helm", bytes.NewBuffer(optdata))
@@ -61,12 +60,12 @@ func Test_helmInstall(t *testing.T) {
is.Equal(http.StatusCreated, rr.Code, "Status should be 201")
body, err := io.ReadAll(rr.Body)
require.NoError(t, err, "ReadAll should not return error")
is.NoError(err, "ReadAll should not return error")
resp := release.Release{}
err = json.Unmarshal(body, &resp)
require.NoError(t, err, "response should be json")
is.Equal(options.Name, resp.Name, "Name doesn't match")
is.Equal(options.Namespace, resp.Namespace, "Namespace doesn't match")
is.NoError(err, "response should be json")
is.EqualValues(options.Name, resp.Name, "Name doesn't match")
is.EqualValues(options.Namespace, resp.Namespace, "Namespace doesn't match")
})
}
+7 -8
View File
@@ -19,7 +19,6 @@ import (
"github.com/segmentio/encoding/json"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_helmList(t *testing.T) {
@@ -28,13 +27,13 @@ func Test_helmList(t *testing.T) {
_, store := datastore.MustNewTestStore(t, true, true)
err := store.Endpoint().Create(&portainer.Endpoint{ID: 1})
require.NoError(t, err, "error creating environment")
assert.NoError(t, err, "error creating environment")
err = store.User().Create(&portainer.User{Username: "admin", Role: portainer.AdministratorRole})
require.NoError(t, err, "error creating a user")
assert.NoError(t, err, "error creating a user")
jwtService, err := jwt.NewService("1h", store)
require.NoError(t, err, "Error initialising jwt service")
is.NoError(err, "Error initialising jwt service")
kubernetesDeployer := exectest.NewKubernetesDeployer()
helmPackageManager := test.NewMockHelmPackageManager()
@@ -57,13 +56,13 @@ func Test_helmList(t *testing.T) {
is.Equal(http.StatusOK, rr.Code, "Status should be 200 OK")
body, err := io.ReadAll(rr.Body)
require.NoError(t, err, "ReadAll should not return error")
is.NoError(err, "ReadAll should not return error")
data := []release.ReleaseElement{}
json.Unmarshal(body, &data)
if is.Len(data, 1, "Expected one chart entry") {
is.Equal(options.Name, data[0].Name, "Name doesn't match")
is.Equal(options.Chart, data[0].Chart, "Chart doesn't match")
if is.Equal(1, len(data), "Expected one chart entry") {
is.EqualValues(options.Name, data[0].Name, "Name doesn't match")
is.EqualValues(options.Chart, data[0].Chart, "Chart doesn't match")
}
})
}
@@ -11,7 +11,6 @@ import (
"github.com/portainer/portainer/pkg/libhelm/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_helmRepoSearch(t *testing.T) {
@@ -34,7 +33,7 @@ func Test_helmRepoSearch(t *testing.T) {
is.Equal(http.StatusOK, rr.Code, "Status should be 200 OK")
body, err := io.ReadAll(rr.Body)
require.NoError(t, err, "ReadAll should not return error")
is.NoError(err, "ReadAll should not return error")
is.NotEmpty(body, "Body should not be empty")
})
}
+3 -5
View File
@@ -10,9 +10,7 @@ import (
"github.com/portainer/portainer/api/internal/testhelpers"
"github.com/portainer/portainer/pkg/libhelm/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_helmShow(t *testing.T) {
@@ -39,11 +37,11 @@ func Test_helmShow(t *testing.T) {
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
is.Equal(http.StatusOK, rr.Code, "Status should be 200 OK")
is.Equal(rr.Code, http.StatusOK, "Status should be 200 OK")
body, err := io.ReadAll(rr.Body)
require.NoError(t, err, "ReadAll should not return error")
is.Equal(string(body), expect, "Unexpected search response")
is.NoError(err, "ReadAll should not return error")
is.EqualValues(string(body), expect, "Unexpected search response")
})
}
}
+1 -2
View File
@@ -4,7 +4,6 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestIsSelfSignedCertificate(t *testing.T) {
@@ -180,7 +179,7 @@ w/pjiPVy
for _, tt := range tc {
t.Run(tt.name, func(t *testing.T) {
actual, err := IsSelfSignedCertificate([]byte(tt.cert))
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, tt.expected, actual)
})
}
+3 -5
View File
@@ -13,9 +13,7 @@ import (
"github.com/portainer/portainer/api/jwt"
"github.com/portainer/portainer/api/kubernetes"
kubeClient "github.com/portainer/portainer/api/kubernetes/cli"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// Currently this test just tests the HTTP Handler is setup correctly, in the future we should move the ClientFactory to a mock in order
@@ -30,13 +28,13 @@ func TestGetKubernetesEvents(t *testing.T) {
Type: portainer.AgentOnKubernetesEnvironment,
},
)
require.NoError(t, err, "error creating environment")
is.NoError(err, "error creating environment")
err = store.User().Create(&portainer.User{Username: "admin", Role: portainer.AdministratorRole})
require.NoError(t, err, "error creating a user")
is.NoError(err, "error creating a user")
jwtService, err := jwt.NewService("1h", store)
require.NoError(t, err, "Error initiating jwt service")
is.NoError(err, "Error initiating jwt service")
tk, _, _ := jwtService.GenerateToken(&portainer.TokenData{ID: 1, Username: "admin", Role: portainer.AdministratorRole})
-1
View File
@@ -105,7 +105,6 @@ func NewHandler(bouncer security.BouncerService, authorizationService *authoriza
endpointRouter.Handle("/cluster_role_bindings", httperror.LoggerHandler(h.getAllKubernetesClusterRoleBindings)).Methods(http.MethodGet)
endpointRouter.Handle("/cluster_role_bindings/delete", httperror.LoggerHandler(h.deleteClusterRoleBindings)).Methods(http.MethodPost)
endpointRouter.Handle("/describe", httperror.LoggerHandler(h.describeResource)).Methods(http.MethodGet)
endpointRouter.Handle("/nodes/{name}/drain", httperror.LoggerHandler(h.drainNode)).Methods(http.MethodPost)
// namespaces
// in the future this piece of code might be in another package (or a few different packages - namespaces/namespace?)
-56
View File
@@ -1,56 +0,0 @@
package kubernetes
import (
"net/http"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
"github.com/portainer/portainer/pkg/libhttp/response"
"github.com/portainer/portainer/pkg/libkubectl"
"github.com/rs/zerolog/log"
)
// @id drainNode
// @summary Drain a Kubernetes node
// @description Drain a Kubernetes node by safely evicting all pods from the node, preparing it for maintenance or removal
// @description **Access policy**: authenticated
// @tags kubernetes
// @security ApiKeyAuth || jwt
// @accept json
// @param id path int true "Environment(Endpoint) identifier"
// @param name path string true "Name of the node to drain"
// @success 204 "Success"
// @failure 400 "Invalid request, such as missing required fields or fields not meeting validation criteria."
// @failure 401 "Unauthorized access - the user is not authenticated or does not have the necessary permissions. Ensure that you have provided a valid API key or JWT token, and that you have the required permissions."
// @failure 403 "Permission denied - the user is authenticated but does not have the necessary permissions to access the requested resource or perform the specified operation. Check your user roles and permissions."
// @failure 404 "Unable to find an environment with the specified identifier or unable to find the specified node."
// @failure 500 "Server error occurred while attempting to drain node."
// @router /kubernetes/{id}/nodes/{name}/drain [post]
func (handler *Handler) drainNode(w http.ResponseWriter, r *http.Request) *httperror.HandlerError {
name, err := request.RetrieveRouteVariableValue(r, "name")
if err != nil {
log.Error().Err(err).Str("context", "drainNode").Msg("Invalid node name route variable")
return httperror.BadRequest("Invalid node name route variable", err)
}
kubeCtlAccess, err := handler.getLibKubectlAccess(r)
if err != nil {
log.Error().Err(err).Str("context", "drainNode").Str("node name", name).Msg("Unable to get a Kubernetes client for the user")
return httperror.InternalServerError("Unable to get a Kubernetes client for the user", err)
}
client, err := libkubectl.NewClient(kubeCtlAccess, "", "", true)
if err != nil {
log.Error().Err(err).Str("context", "drainNode").Msg("Failed to create kubernetes client")
return httperror.InternalServerError("an error occurred during the drainNode operation, failed to create kubernetes client. Error: ", err)
}
output, err := client.DrainNode(name)
if err != nil {
log.Error().Err(err).Str("context", "drainNode").Msg("Failed to drain node")
return httperror.InternalServerError("an error occurred during the drainNode operation, failed to drain node. Error: ", err)
}
log.Debug().Str("context", "drainNode").Str("node name", name).Str("output", output).Msg("Drained node")
return response.Empty(w)
}

Some files were not shown because too many files have changed in this diff Show More