Skip to content

Commit d426db2

Browse files
committed
feat: Add OCI artifact support for runtime configurations with clean architecture
This commit implements comprehensive OCI artifact support for exporting and importing runtime configurations, enabling users to store and share ToolHive configurations as OCI artifacts in container registries. The implementation follows clean architecture principles with proper separation of concerns. - **Export to OCI**: Export runtime configurations as OCI artifacts to registries - **Import from OCI**: Automatically detect and load OCI runtime configuration artifacts - **Registry-Only Operations**: OCI artifacts stored exclusively in registries (no local daemon) - **Automatic Detection**: Smart detection of OCI references vs file paths - Extended `thv export` to support both file paths and OCI references - Automatic detection of OCI references vs file paths - Maintained backward compatibility with existing file-based exports - Clear messaging: "to file" vs "to OCI registry" - Modified `thv run` to automatically detect OCI runtime configuration artifacts - Graceful fallback when OCI references aren't runtime config artifacts - Seamless integration with existing container image and registry workflows - **`pkg/oci/`** - General OCI registry operations and utilities - `client.go`: Generic OCI client with authentication - `utils.go`: Reference validation, tag extraction, utilities - Reusable across different artifact types - **`pkg/runner/export/`** - RunConfig-specific export/import operations - `oci.go`: Specialized OCI operations for runtime configurations - Custom media types and annotations for RunConfig artifacts - ORAS-based push/pull operations - **OCI Client** (`pkg/oci/client.go`): - `CreateRepository()`: Authenticated ORAS repository clients - Authentication bridge using existing keychain infrastructure - Generic, reusable for any OCI artifact operations - **OCI Utilities** (`pkg/oci/utils.go`): - `ValidateReference()`: OCI reference validation - `IsOCIReference()`: Smart detection of OCI vs file references - `ExtractTag()`: Tag parsing utilities - **RunConfig Exporter** (`pkg/runner/export/oci.go`): - `PushRunConfig()`: Push runtime configurations to registries - `PullRunConfig()`: Pull runtime configurations from registries - Custom media types: `application/vnd.toolhive.runconfig.v1+json` - Proper OCI annotations (creation time, version, description) Following ORAS best practices, OCI artifacts are stored exclusively in registries: - No local daemon storage for OCI artifacts - Direct registry operations using ORAS - Simplified architecture: OCI reference → registry, file path → local file - Leverages existing go-containerregistry keychain infrastructure - Secure handling of registry credentials - Proper validation of OCI references and artifacts - Input validation for all new parameters - **Authentication Errors**: Proper detection and reporting of auth failures - **Network Errors**: Distinction between network issues and artifact problems - **Artifact Type Errors**: Graceful handling when OCI refs aren't runtime configs - Clear error messages for different failure scenarios - Automatic fallback behavior for ambiguous references - **Export Command** (`cmd/thv/app/export.go`): - Support for OCI references alongside file paths - Early validation using `oci.ValidateReference()` - Improved error handling and user messaging - **Run Command** (`cmd/thv/app/run.go`): - Updated help text to document OCI runtime configuration support - Integration with retriever for automatic OCI artifact detection - **Retriever** (`pkg/runner/retriever/retriever.go`): - Enhanced `GetMCPServer()` to try OCI artifacts before container images - Intelligent error handling to distinguish artifact types - Graceful fallback for non-runtime-config OCI references - **Run Flags** (`cmd/thv/app/run_flags.go`): - Modified `BuildRunnerConfig()` to detect OCI runtime configurations - Seamless integration with existing configuration loading ```bash thv export my-server ./config.json thv export my-server registry.example.com/configs/my-server:latest ``` ```bash thv run registry.example.com/configs/my-server:latest thv run registry.example.com/regular-image:latest ``` - Added `oras.land/oras-go/v2` for OCI artifact operations - Leveraged existing `github.com/google/go-containerregistry` for reference validation - Clean integration with existing authentication infrastructure - **Comprehensive Test Coverage**: - OCI package tests for general operations - Export package tests for RunConfig-specific functionality - Retriever tests for OCI artifact detection and loading - Integration tests for end-to-end workflows - **Code Quality**: - All linting issues resolved - Proper error handling and logging - Clean separation of concerns - Comprehensive documentation - Updated CLI help text for `thv export` and `thv run` commands - Updated documentation files (`docs/cli/thv_export.md`, `docs/cli/thv.md`) - Added comprehensive examples and usage patterns - All existing functionality remains unchanged - File-based export/import continues to work as before - No breaking changes to existing CLI interfaces - Graceful degradation when OCI features are not available This implementation extends ToolHive's capabilities while maintaining its core principles of simplicity, security, and reliability. The clean architecture ensures the codebase remains maintainable and extensible for future OCI artifact types. Signed-off-by: Juan Antonio Osorio <ozz@stacklok.com>
1 parent 7f3beeb commit d426db2

File tree

18 files changed

+1398
-21
lines changed

18 files changed

+1398
-21
lines changed

cmd/thv/app/export.go

Lines changed: 57 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,93 @@
11
package app
22

33
import (
4+
"context"
45
"fmt"
56
"os"
67
"path/filepath"
78

89
"github.com/spf13/cobra"
910

11+
"github.com/stacklok/toolhive/pkg/container/images"
12+
"github.com/stacklok/toolhive/pkg/oci"
1013
"github.com/stacklok/toolhive/pkg/runner"
14+
"github.com/stacklok/toolhive/pkg/runner/export"
1115
)
1216

1317
func newExportCmd() *cobra.Command {
14-
return &cobra.Command{
15-
Use: "export <workload name> <path>",
16-
Short: "Export a workload's run configuration to a file",
17-
Long: `Export a workload's run configuration to a file for sharing or backup.
18+
cmd := &cobra.Command{
19+
Use: "export <workload name> <path|oci-reference>",
20+
Short: "Export a workload's run configuration to a file or OCI artifact",
21+
Long: `Export a workload's run configuration to a file or OCI artifact for sharing or backup.
1822
19-
The exported configuration can be used with 'thv run --from-config <path>' to recreate
20-
the same workload with identical settings.
23+
The exported configuration can be used with 'thv run --from-config <path>' or
24+
'thv run --from-config <oci-reference>' to recreate the same workload with identical settings.
25+
26+
When exporting to an OCI reference, the artifact is pushed to the remote registry.
27+
When exporting to a file path, the configuration is saved as a JSON file.
2128
2229
Examples:
2330
# Export a workload configuration to a file
2431
thv export my-server ./my-server-config.json
2532
2633
# Export to a specific directory
27-
thv export github-mcp /tmp/configs/github-config.json`,
34+
thv export github-mcp /tmp/configs/github-config.json
35+
36+
# Export to an OCI artifact (pushes to registry)
37+
thv export my-server registry.example.com/configs/my-server:latest`,
2838
Args: cobra.ExactArgs(2),
2939
RunE: exportCmdFunc,
3040
}
41+
42+
return cmd
3143
}
3244

3345
func exportCmdFunc(cmd *cobra.Command, args []string) error {
3446
ctx := cmd.Context()
3547
workloadName := args[0]
36-
outputPath := args[1]
48+
destination := args[1]
3749

3850
// Load the saved run configuration
3951
runnerInstance, err := runner.LoadState(ctx, workloadName)
4052
if err != nil {
4153
return fmt.Errorf("failed to load run configuration for workload '%s': %w", workloadName, err)
4254
}
4355

56+
// Determine if destination is an OCI reference or file path
57+
if oci.IsOCIReference(destination) {
58+
return exportToOCI(ctx, runnerInstance.Config, workloadName, destination)
59+
}
60+
61+
return exportToFile(runnerInstance.Config, workloadName, destination)
62+
}
63+
64+
// exportToOCI exports the configuration to an OCI artifact
65+
func exportToOCI(ctx context.Context, config *runner.RunConfig, workloadName, ref string) error {
66+
// Validate the OCI reference early
67+
if err := oci.ValidateReference(ref); err != nil {
68+
return fmt.Errorf("invalid OCI reference: %w", err)
69+
}
70+
71+
imageManager := images.NewImageManager(ctx)
72+
ociClient := oci.NewClient(imageManager)
73+
exporter := export.NewOCIExporter(ociClient)
74+
75+
// Push directly to registry (no local storage for OCI artifacts)
76+
if err := exporter.PushRunConfig(ctx, config, ref); err != nil {
77+
return fmt.Errorf("failed to push configuration to OCI registry: %w", err)
78+
}
79+
fmt.Printf("Successfully exported run configuration for '%s' to OCI registry '%s'\n", workloadName, ref)
80+
81+
return nil
82+
}
83+
84+
// exportToFile exports the configuration to a local file
85+
func exportToFile(config *runner.RunConfig, workloadName, outputPath string) error {
86+
// Validate input
87+
if config == nil {
88+
return fmt.Errorf("configuration cannot be nil")
89+
}
90+
4491
// Ensure the output directory exists
4592
outputDir := filepath.Dir(outputPath)
4693
if err := os.MkdirAll(outputDir, 0750); err != nil {
@@ -56,10 +103,10 @@ func exportCmdFunc(cmd *cobra.Command, args []string) error {
56103
defer outputFile.Close()
57104

58105
// Write the configuration to the file
59-
if err := runnerInstance.Config.WriteJSON(outputFile); err != nil {
106+
if err := config.WriteJSON(outputFile); err != nil {
60107
return fmt.Errorf("failed to write configuration to file: %w", err)
61108
}
62109

63-
fmt.Printf("Successfully exported run configuration for '%s' to '%s'\n", workloadName, outputPath)
110+
fmt.Printf("Successfully exported run configuration for '%s' to file '%s'\n", workloadName, outputPath)
64111
return nil
65112
}

cmd/thv/app/export_test.go

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
package app
2+
3+
import (
4+
"testing"
5+
6+
"github.com/stretchr/testify/assert"
7+
8+
"github.com/stacklok/toolhive/pkg/runner"
9+
)
10+
11+
func TestExportToFile_InvalidPath(t *testing.T) {
12+
t.Parallel()
13+
14+
// Create a valid config for testing
15+
config := &runner.RunConfig{
16+
Name: "test-config",
17+
Image: "test-image:latest",
18+
}
19+
20+
// Test with invalid directory path
21+
err := exportToFile(config, "test", "/invalid/path/that/does/not/exist/config.json")
22+
assert.Error(t, err)
23+
assert.Contains(t, err.Error(), "failed to create output directory")
24+
}
25+
26+
func TestExportToFile_NilConfig(t *testing.T) {
27+
t.Parallel()
28+
29+
// Test with nil config
30+
err := exportToFile(nil, "test", "/tmp/test-config.json")
31+
assert.Error(t, err)
32+
}

cmd/thv/app/run.go

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ var runCmd = &cobra.Command{
2424
Short: "Run an MCP server",
2525
Long: `Run an MCP server with the specified name, image, or protocol scheme.
2626
27-
ToolHive supports four ways to run an MCP server:
27+
ToolHive supports five ways to run an MCP server:
2828
2929
1. From the registry:
3030
$ thv run server-name [-- args...]
@@ -45,7 +45,12 @@ ToolHive supports four ways to run an MCP server:
4545
or go (Golang). For Go, you can also specify local paths starting
4646
with './' or '../' to build and run local Go projects.
4747
48-
4. From an exported configuration:
48+
4. From an OCI runtime configuration artifact:
49+
$ thv run registry.example.com/configs/my-server:latest
50+
Runs an MCP server using a runtime configuration stored as an OCI artifact.
51+
The system automatically detects and loads the configuration.
52+
53+
5. From an exported configuration:
4954
$ thv run --from-config <path>
5055
Runs an MCP server using a previously exported configuration file.
5156

cmd/thv/app/run_flags.go

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,14 @@ import (
88

99
cfg "github.com/stacklok/toolhive/pkg/config"
1010
"github.com/stacklok/toolhive/pkg/container"
11+
"github.com/stacklok/toolhive/pkg/container/images"
1112
"github.com/stacklok/toolhive/pkg/container/runtime"
13+
"github.com/stacklok/toolhive/pkg/logger"
14+
"github.com/stacklok/toolhive/pkg/oci"
1215
"github.com/stacklok/toolhive/pkg/process"
1316
"github.com/stacklok/toolhive/pkg/registry"
1417
"github.com/stacklok/toolhive/pkg/runner"
18+
"github.com/stacklok/toolhive/pkg/runner/export"
1519
"github.com/stacklok/toolhive/pkg/runner/retriever"
1620
"github.com/stacklok/toolhive/pkg/transport"
1721
"github.com/stacklok/toolhive/pkg/transport/types"
@@ -317,3 +321,59 @@ func getTelemetryFromFlags(cmd *cobra.Command, config *cfg.Config, otelEndpoint
317321

318322
return finalOtelEndpoint, finalOtelSamplingRate, finalOtelEnvironmentVariables
319323
}
324+
325+
// isOCIRuntimeConfigArtifact checks if the serverOrImage is an OCI runtime configuration artifact
326+
func isOCIRuntimeConfigArtifact(ctx context.Context, serverOrImage string) bool {
327+
// Try to parse as OCI reference first
328+
if !oci.IsOCIReference(serverOrImage) {
329+
return false
330+
}
331+
332+
// Try to load as runtime configuration
333+
imageManager := images.NewImageManager(ctx)
334+
ociClient := oci.NewClient(imageManager)
335+
exporter := export.NewOCIExporter(ociClient)
336+
_, err := exporter.PullRunConfig(ctx, serverOrImage)
337+
return err == nil
338+
}
339+
340+
// loadAndMergeOCIRunConfig loads a runtime configuration from OCI and merges with command-line flags
341+
func loadAndMergeOCIRunConfig(ctx context.Context, ref string, flags *RunFlags, debugMode bool) (*runner.RunConfig, error) {
342+
// Load the runtime configuration from OCI
343+
imageManager := images.NewImageManager(ctx)
344+
ociClient := oci.NewClient(imageManager)
345+
exporter := export.NewOCIExporter(ociClient)
346+
ociConfig, err := exporter.PullRunConfig(ctx, ref)
347+
if err != nil {
348+
return nil, fmt.Errorf("failed to load runtime configuration from OCI artifact: %w", err)
349+
}
350+
351+
// Create container runtime
352+
rt, err := container.NewFactory().Create(ctx)
353+
if err != nil {
354+
return nil, fmt.Errorf("failed to create container runtime: %v", err)
355+
}
356+
357+
// Set the runtime in the config
358+
ociConfig.Deployer = rt
359+
360+
// Override with any command-line flags that were explicitly set
361+
// This allows users to override specific settings from the OCI config
362+
if flags.Name != "" {
363+
ociConfig.Name = flags.Name
364+
}
365+
if flags.Host != "" {
366+
ociConfig.Host = flags.Host
367+
}
368+
if flags.ProxyPort != 0 {
369+
ociConfig.Port = flags.ProxyPort
370+
}
371+
if debugMode {
372+
ociConfig.Debug = true
373+
}
374+
375+
logger.Infof("Successfully loaded runtime configuration from OCI artifact: %s", ref)
376+
logger.Infof("Using image: %s", ociConfig.Image)
377+
378+
return ociConfig, nil
379+
}

0 commit comments

Comments
 (0)