Merge branch 'dle-4-0' into 'master'

feat(engine): 4.0 alpha – snapshots on demand, Git-like branches, Apache 2.0 license, rename DLE→DBLab

See merge request postgres-ai/database-lab!695

Author: Bogdan Tsechoev

.gitignore

@@ -1,5 +1,6 @@
 .DS_Store
 .idea/
+.env
 
 engine/bin/
 /db-lab-run/

CLAUDE.md

@@ -0,0 +1,23 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Build/Test/Lint Commands
+- Build all components: `cd engine && make build`
+- Lint code: `cd engine && make lint`
+- Run unit tests: `cd engine && make test`
+- Run integration tests: `cd engine && make test-ci-integration`
+- Run a specific test: `cd engine && GO111MODULE=on go test -v ./path/to/package -run TestName`
+- Run UI: `cd ui && pnpm start:ce` (Community Edition) or `pnpm start:platform`
+
+## Code Style Guidelines
+- Go code follows "Effective Go" and "Go Code Review Comments" guidelines
+- Use present tense and imperative mood in commit messages
+- Limit first commit line to 72 characters
+- All Git commits must be signed
+- Format Go code with `cd engine && make fmt`
+- Use error handling with pkg/errors
+- Follow standard Go import ordering
+- Group similar functions together
+- Error messages should be descriptive and actionable
+- UI uses pnpm for package management

CONTRIBUTING.md

@@ -23,11 +23,11 @@ These are mostly guidelines, not rules. Use your best judgment, and feel free to
     - [Git commit messages](#git-commit-messages)
     - [Go styleguide](#go-styleguide)
     - [Documentation styleguide](#documentation-styleguide)
+    - [API design and testing](#api-design-and-testing)
+    - [UI development](#ui-development)
     - [Development setup](#development-setup)
     - [Repo overview](#repo-overview)
-<!--
     - [Building from source](#building-from-source)
--->
 
 ---
 
@@ -121,6 +121,45 @@ We encourage you to follow the principles described in the following documents:
 - [Effective Go](https://go.dev/doc/effective_go)
 - [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments)
 
+### Message style guide
+Consistent messaging is important throughout the codebase. Follow these guidelines for errors, logs, and user-facing messages:
+
+#### Error messages
+- Lowercase for internal errors and logs: `failed to start session` (no ending period)
+- Uppercase for user-facing errors: `Requested object does not exist. Specify your request.` (with ending period)
+- Omit articles ("a", "an", "the") for brevity: use `failed to update clone` not `failed to update the clone`
+- Be specific and actionable whenever possible
+- For variable interpolation, use consistent formatting: `failed to find clone: %s`
+
+#### CLI output
+- Use concise, action-oriented language
+- Present tense with ellipsis for in-progress actions: `Creating clone...` 
+  - Ellipsis (`...`) indicates an ongoing process where the user should wait
+  - Always follow up with a completion message when the operation finishes
+- Past tense with period for results: `Clone created successfully.`
+- Include relevant identifiers (IDs, names) in output
+
+#### Progress indication
+- Use ellipsis (`...`) to indicate that an operation is in progress and the user should wait
+- For longer operations, consider providing percentage or step indicators: `Cloning database... (25%)`
+- When an operation with ellipsis completes, always provide a completion message without ellipsis
+- Example sequence:
+  ```
+  Creating clone...
+  Clone "test-clone" created successfully.
+  ```
+
+#### UI messages
+- Be consistent with terminology across UI and documentation
+- For confirmations, use format: `{Resource} {action} successfully.`
+- For errors, provide clear next steps when possible
+- Use sentence case for all messages (capitalize first word only)
+
+#### Commit messages
+- Start with lowercase type prefix: `fix:`, `feat:`, `docs:`, etc.
+- Use imperative mood: `add feature` not `added feature`
+- Provide context in the body if needed
+
 ### Documentation styleguide
 Documentation for Database Lab Engine and additional components is hosted at https://postgres.ai/docs and is maintained in this GitLab repo: https://gitlab.com/postgres-ai/docs.
 
@@ -132,6 +171,94 @@ We're building documentation following the principles described at https://docum
 
 Learn more: https://documentation.divio.com/.
 
+### API design and testing
+The DBLab API follows RESTful principles with these key guidelines:
+- Clear resource-based URL structure
+- Consistent usage of HTTP methods (GET, POST, DELETE, etc.)
+- Standardized error responses
+- Authentication via API tokens
+- JSON for request and response bodies
+- Comprehensive documentation with examples
+
+#### API Documentation
+We use readme.io to host the API docs: https://dblab.readme.io/ and https://api.dblab.dev.
+
+When updating the API specification:
+1. Make changes to the OpenAPI spec file in `engine/api/swagger-spec/`
+2. Upload it to readme.io as a new documentation version
+3. Review and publish the new version
+
+#### Testing with Postman and Newman
+Postman collection is generated based on the OpenAPI spec file, using [Portman](https://github.com/apideck-libraries/portman).
+
+##### Setup and Generation
+1. Install Portman: `npm install -g @apideck/portman`
+2. Generate Postman collection file:
+   ```
+   portman --cliOptionsFile engine/api/postman/portman-cli.json
+   ```
+
+##### Test Structure Best Practices
+- Arrange tests in logical flows (create, read, update, delete)
+- Use environment variables to store and pass data between requests
+- For object creation tests, capture the ID in the response to use in subsequent requests
+- Add validation tests for response status, body structure, and expected values
+- Clean up created resources at the end of test flows
+
+##### CI/CD Integration
+The Postman collection is automatically run in CI/CD pipelines using Newman. For local testing:
+```
+newman run engine/api/postman/dblab_api.postman_collection.json -e engine/api/postman/branching.aws.postgres.ai.postman_environment.json
+```
+
+### UI development
+The Database Lab Engine UI contains two main packages:
+- `@postgres.ai/platform` - Platform version of UI
+- `@postgres.ai/ce` - Community Edition version of UI
+- `@postgres.ai/shared` - Common modules shared between packages
+
+#### Working with UI packages
+At the repository root:
+- `pnpm install` - Install all dependencies
+- `npm run build -ws` - Build all packages
+- `npm run start -w @postgres.ai/platform` - Run Platform UI in dev mode
+- `npm run start -w @postgres.ai/ce` - Run Community Edition UI in dev mode
+
+_Note: Don't use commands for `@postgres.ai/shared` - it's a dependent package that can't be run or built directly_
+
+#### Platform UI Development
+1. Set up environment variables:
+   ```bash
+   cd ui/packages/platform
+   cp .env_example_dev .env
+   ```
+2. Edit `.env` to set:
+   - `REACT_APP_API_URL_PREFIX` to point to dev API server
+   - `REACT_APP_TOKEN_DEBUG` to set your JWT token
+3. Start development server: `pnpm run start`
+
+#### CI pipelines for UI code
+To deploy UI changes, tag the commit with `ui/` prefix and push it:
+```shell
+git tag ui/1.0.12
+git push origin ui/1.0.12
+```
+
+#### Handling Vulnerabilities
+When addressing vulnerabilities in UI packages:
+1. Update the affected package to a newer version if available
+2. For sub-package vulnerabilities, try using [npm-force-resolutions](https://www.npmjs.com/package/npm-force-resolutions)
+3. As a last resort, consider forking the package locally
+
+For code-related issues:
+1. Consider rewriting JavaScript code in TypeScript
+2. Follow recommendations from security analysis tools
+3. Only ignore false positives when absolutely necessary
+
+#### TypeScript Migration
+- `@postgres.ai/shared` and `@postgres.ai/ce` are written in TypeScript
+- `@postgres.ai/platform` is partially written in TypeScript with ongoing migration efforts
+
 ### Repo overview
 The [postgres-ai/database-lab](https://gitlab.com/postgres-ai/database-lab) repo contains 2 components:
 - [Database Lab Engine](https://gitlab.com/postgres-ai/database-lab/-/tree/master/engine)
@@ -190,10 +317,27 @@ Components have a separate version, denoted by either:
 
 
 ### Building from source
-Use `Makefile` to build Database Lab components from source.
+The Database Lab Engine provides multiple build targets in its `Makefile`:
+
+```bash
+cd engine
+make help      # View all available build targets
+make build     # Build all components (Server, CLI, CI Checker)
+make build-dle # Build Database Lab Engine binary and Docker image
+make test      # Run unit tests
+```
+
+You can also build specific components:
+
+```bash
+# Build the CLI for all supported platforms
+make build-client
+
+# Build the Server in debug mode
+make build-debug
 
-Run `make help` to see all available targets.
+# Build and run DLE locally
+make run-dle
+```
 
-<!--
-mention dev images: See our [GitLab Container Registry](https://gitlab.com/postgres-ai/database-lab/container_registry) to find the images built for development branches.
--->
+See our [GitLab Container Registry](https://gitlab.com/postgres-ai/database-lab/container_registry) to find pre-built images for development branches.

README.md

@@ -8,18 +8,18 @@
 <div align="center"><h1 align="center">DBLab Engine</h1></div>
 
 <div align="center">
-  <a href="https://twitter.com/intent/tweet?via=Database_Lab&url=https://github.com/postgres-ai/database-lab-engine/&text=20@PostgreSQL%branching%20–%20DLE%20provides%20blazing-fast%20database%20cloning%20to%20build%20powerful%20development,%20test,%20QA,%20staging%20environments.">
+  <a href="https://twitter.com/intent/tweet?via=Database_Lab&url=https://github.com/postgres-ai/database-lab-engine/&text=PostgreSQL%20branching%20%E2%80%93%20DLE%20provides%20blazing-fast%20database%20cloning%20to%20build%20powerful%20development,%20test,%20QA,%20and%20staging%20environments.">
     <img src="https://img.shields.io/twitter/url/https/github.com/postgres-ai/database-lab-engine.svg?style=for-the-badge" alt="twitter">
   </a>
 </div>
 
 <div align="center">
-  <strong>⚡ Blazing-fast Postgres cloning and branching 🐘</strong><br /><br />
+  <strong>⚡ Blazing-fast PostgreSQL cloning and branching 🐘</strong><br /><br />
   🛠️ Build powerful dev/test environments.<br />
   🔃 Cover 100% of DB migrations with CI tests.<br>
   💡 Quickly verify ChatGPT ideas to get rid of hallucinations.<br /><br />
-  Available for any PostgreSQL, including self-managed and managed<sup>*</sup> like AWS RDS, GCP CloudSQL, Supabase, Timescale.<br /><br />
-  Can be installed and used anywhere: all clouds and on-premises.
+  Available for any PostgreSQL, including self-managed and managed services<sup>*</sup> like AWS RDS, GCP Cloud SQL, Supabase, and Timescale.<br /><br />
+  It can be installed and used anywhere: across all cloud environments and on-premises.
 </div>
 
 <br />
@@ -60,11 +60,11 @@ For example, cloning a 1 TiB PostgreSQL database takes just about 10 seconds. On
 <p><img src="./assets/dle-demo-animated.gif" border="0" /></p>
 
 Try it yourself right now:
-- Visit [Postgres.ai Console](https://console.postgres.ai/), set up your first organization and provision a DBLab Standard Edition (DBLab SE) to any cloud or on-prem
+- Visit [Postgres.ai Console](https://console.postgres.ai/), set up your first organization, and provision a DBLab Standard Edition (DBLab SE) to any cloud or on-premises environment.
     - [Pricing](https://postgres.ai/pricing) (starting at $62/month)
-    - [Doc: How to install DBLab SE](https://postgres.ai/docs/how-to-guides/administration/install-dle-from-postgres-ai)
+    - [Documentation: How to install DBLab SE](https://postgres.ai/docs/how-to-guides/administration/install-dle-from-postgres-ai)
 - Demo: https://demo.dblab.dev (use the token `demo-token` to access)
-- Looking for a free version? Install DBLab Community Edition by [following this tutorial](https://postgres.ai/docs/tutorials/database-lab-tutorial)
+- Looking for a free version? Install the DBLab Community Edition by [following this tutorial](https://postgres.ai/docs/tutorials/database-lab-tutorial).
 
 ## How it works
 Thin cloning is fast because it is based on [Copy-on-Write (CoW)](https://en.wikipedia.org/wiki/Copy-on-write#In_computer_storage). DBLab employs two technologies for enabling thin cloning: [ZFS](https://en.wikipedia.org/wiki/ZFS) (default) and [LVM](https://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux)).
@@ -88,20 +88,20 @@ Read more:
 
 ## Features
 - Speed & scale
-    - Blazing-fast cloning of Postgres databases – clone in seconds, irrespective of database size
+    - Blazing-fast cloning of PostgreSQL databases – clone in seconds, irrespective of database size
     - Theoretical max of snapshots/clones: 2<sup>64</sup> ([ZFS](https://en.wikipedia.org/wiki/ZFS), default)
     - Maximum size of PostgreSQL data directory: 256 quadrillion zebibytes, or 2<sup>128</sup> bytes ([ZFS](https://en.wikipedia.org/wiki/ZFS), default)
 - Support & technologies
     - Supported PostgreSQL versions: 9.6–17
     - Thin cloning ([CoW](https://en.wikipedia.org/wiki/Copy-on-write)) technologies: [ZFS](https://en.wikipedia.org/wiki/ZFS) and [LVM](https://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux))
     - UI for manual tasks and API & CLI for automation
     - Packaged in Docker containers for all components
-- Postgres containers
+- PostgreSQL containers
     - Popular extensions including contrib modules, pgvector, HypoPG and many others ([docs](https://postgres.ai/docs/database-lab/supported-databases#extensions-included-by-default))
     - Customization capabilities for containers ([docs](https://postgres.ai/docs/database-lab/supported-databases#how-to-add-more-extensions))
-    - Docker container and Postgres config parameters in DBLab config
+    - Docker container and PostgreSQL configuration parameters in the DBLab config
 - Source database requirements
-    - Location flexibility: self-managed Postgres, AWS RDS, GCP CloudSQL, Azure, etc. No source adjustments needed
+    - Location flexibility: self-managed PostgreSQL, AWS RDS, GCP Cloud SQL, Azure, etc.—no source adjustments needed.
     - No ZFS or Docker requirements for source databases
 - Data provisioning & retrieval
     - Physical (pg_basebackup, WAL-G, pgBackRest) and logical (dump/restore) provisioning
@@ -128,8 +128,8 @@ The simplest way to show your support is by giving us a star on GitHub or GitLab
 ![Add a star](./assets/star.gif)
 
 ### Spread the word
-- Shoot out a tweet and mention [@Database_Lab](https://twitter.com/Database_Lab) 
-- Share this repo's link on your favorite social media platform
+- Tweet about DBLab and mention [@Database_Lab](https://twitter.com/Database_Lab).
+- Share a link to this repository on your favorite social media platform.
 
 ### Share your experience
 If DBLab has been a vital tool for you, tell the world about your journey. Use the logo from the `./assets` folder for a visual touch. Whether it's in documents, presentations, applications, or on your website, let everyone know you trust and use DBLab.
@@ -157,10 +157,7 @@ For darker backgrounds:
 ```
 
 ### Propose an idea or report a bug
-Check out our [contributing guide](./CONTRIBUTING.md) for more details.
-
-### Participate in development
-Check out our [contributing guide](./CONTRIBUTING.md) for more details.
+For proposals, bug reports, and participation in development, see our [Contributing Guide](./CONTRIBUTING.md).
 
 
 ### Reference guides
@@ -173,8 +170,11 @@ Check out our [contributing guide](./CONTRIBUTING.md) for more details.
 - [How to install and initialize Database Lab CLI](https://postgres.ai/docs/how-to-guides/cli/cli-install-init)
 - [How to manage DBLab](https://postgres.ai/docs/how-to-guides/administration)
 - [How to work with clones](https://postgres.ai/docs/how-to-guides/cloning)
+- [How to work with branches](XXXXXXX) – TBD
+- [How to integrate DBLab with GitHub Actions](XXXXXXX) – TBD
+- [How to integrate DBLab with GitLab CI/CD](XXXXXXX) – TBD
 
-More you can find in [the "How-to guides" section](https://postgres.ai/docs/how-to-guides) of the docs. 
+You can find more in the ["How-to guides" section](https://postgres.ai/docs/how-to-guides) of the documentation.
 
 ### Miscellaneous
 - [DBLab Docker images](https://hub.docker.com/r/postgresai/dblab-server)
@@ -183,15 +183,15 @@ More you can find in [the "How-to guides" section](https://postgres.ai/docs/how-
 - [DB Migration Checker](https://postgres.ai/docs/db-migration-checker)
 
 ## License
-DBLab source code is licensed under the OSI-approved open source license [Apache 2.0](https://opensource.org/license/apache-2-0/).
+The DBLab source code is licensed under the OSI-approved open source license [Apache 2.0](https://opensource.org/license/apache-2-0/).
 
 Reach out to the Postgres.ai team if you want a trial or commercial license that does not contain the GPL clauses: [Contact page](https://postgres.ai/contact).
 
 ## Community & Support
-- ["Database Lab Engine Community Covenant Code of Conduct"](./CODE_OF_CONDUCT.md)
-- Where to get help: [Contact page](https://postgres.ai/contact)
+- [Database Lab Engine Community Covenant Code of Conduct](./CODE_OF_CONDUCT.md)
+- Where to get help: [Contact page](https://postgres.ai/contact).
 - [Community Slack](https://slack.postgres.ai)
-- If you need to report a security issue, follow instructions in ["Database Lab Engine security guidelines"](./SECURITY.md)
+- If you need to report a security issue, follow the instructions in [Database Lab Engine Security Guidelines](./SECURITY.md).
 
 [![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg?color=blue)](./CODE_OF_CONDUCT.md)