Add + Reorganise SQLMesh Tests #27
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add new directories to .gitignore for Dagster project storage, logs, history, and schedules. - Modify Makefile to support Windows environments and add new clean commands for Dagster and database. - Update pyproject.toml to change version to 0.8.0 and add new dependencies for FastAPI and SSE-Starlette. - Refactor type hints across multiple files to use modern syntax and improve consistency. - Enhance console and event handling in dagster_sqlmesh for better logging and event management. - Clean up imports and ensure proper typing throughout the codebase. - Update sample project definitions for compatibility with new changes.
- Update type hints to use modern syntax throughout the file. - Clean up and reorganize import statements for better clarity. - Enhance function signatures with appropriate return types. - Ensure consistency in typing for fixtures and context classes.
- Unified command definitions for Python, SQLMESH, and UV to streamline setup. - Removed Windows-specific command handling for a cleaner and more consistent configuration. - Simplified activation and deactivation commands for the virtual environment.
- Simplified assertion statements in the test_restating_models function for better clarity. - Enhanced readability by formatting assertions with inline messages. - Maintained existing test logic and functionality.
- Deleted `conftest.py`, `test_asset.py`, and `test_sqlmesh_context.py` as they were no longer needed. - Cleaned up the codebase by removing obsolete test cases and associated fixtures. - Ensured that the remaining tests and functionality are unaffected by these deletions. - Added new tests
- Introduced `permanent_sqlmesh_project` fixture to provide a stable sample SQLMesh project for testing. - Updated `model_change_test_context` to utilize the new permanent project fixture for improved debugging. - Added a new row to `seed_data_1.csv` for enhanced test coverage. - Marked `test_given_model_chain_when_running_with_different_flags_then_behaves_as_expected` as a work-in-progress. - Commented out unused code in the test to focus on current implementation.
- Change GROUP BY clause in `full_model.sql` to use `GROUP BY ALL`. - Update docstrings in `conftest.py` to indicate debugging context for fixtures. - Uncomment and utilize `initialize_test_source` in `model_change_test_context`. - Replace `model_change_test_context` with `sample_sqlmesh_test_context` in tests for consistency. - Remove commented-out code in `test_model_code_change.py` to enhance clarity and focus on current implementation.
- Change SQL model definition to use lowercase for `kind` and uppercase for `NAME`. - Add `item_name` to the SELECT statement and include `last_updated_at` in the output. - Update test configuration to exclude `last_updated_at` and mark the output as partial. - Improve test assertions by comparing the full model DataFrame against an expected DataFrame. - Ensure consistency in test context usage and enhance clarity in test logic.
- Add a comment to clarify the reason for skipping tests due to outdated `test_full_model.yaml`. - Remove unnecessary sorting and resetting of index in the test to streamline the DataFrame processing.
- Store initial and post-modification timestamps for `last_updated_at` to verify changes. - Add assertions to ensure timestamps reflect expected behavior after model modifications and when no changes are made. - Improve clarity and context in the test by including comments on the purpose of timestamp checks.
- Deleted `conftest.py`, `test_asset.py`, `test_sqlmesh_context.py`, and other related test files as they are no longer needed. - Ensured that the remaining tests and functionality remain unaffected by these deletions. - Improved overall codebase clarity by removing obsolete test cases and associated fixtures.
- Included `pandas-stubs` in both `pyproject.toml` and `uv.lock` for improved type hinting support with pandas. - Updated `conftest.py` to specify type hints for `_backed_up_files` as a set of strings. - Simplified type hinting in `resource.py` by initializing `_count` without explicit type declaration.
- Removed the `no_auto_upstream` parameter from the test function to streamline its signature. - Updated the parameterized test case to focus on `skip_backfill` and `expected_changes`. - Enhanced the test ID for better readability and understanding of the test's purpose.
- Introduced `DagsterTestContext` for improved asset materialization in tests. - Updated `sample_sqlmesh_project` and `sample_dagster_project` fixtures to create temporary projects for testing. - Modified `SQLMeshResource` to support plan and run options overrides. - Refactored existing test cases to utilize the new context and improve clarity. - Updated `Makefile` to change asset materialization command and configuration for better usability.
- Renamed `sample_sqlmesh_project` to `sample_project_root` for clarity and improved fixture structure. - Introduced `sample_dagster_project` fixture to streamline project setup for Dagster tests. - Added `_run_command` method in `DagsterTestContext` for real-time command execution and output streaming. - Updated `pytest.main` invocation in test files to include verbose and capture options for better output during test runs. - Cleaned up temporary project creation logic to ensure proper directory handling and resource management.
…reaming - Refactored `_run_command` method in `DagsterTestContext` to utilize threading and queues for real-time output streaming from subprocesses. - Added a helper function `stream_output` to handle output from stdout and stderr concurrently. - Updated test file to utilize `asset_materialisation` method for asset handling, improving clarity in test setup. - Removed commented-out code to clean up the test file.
- Added functionality to save the current SQLMesh project state to a timestamped directory within the 'debug_runs' folder. - Enhanced `sample_project_root` fixture to create a debug directory after test runs for easier debugging. - Updated `DagsterTestContext` to include separate paths for Dagster and SQLMesh projects, improving clarity and organization. - Modified test cases to utilize the new debug state saving feature, facilitating better tracking of project states during tests. - Adjusted paths in `definitions.py` to ensure correct database file location relative to the SQLMesh project.
- Updated `_run_command` method to accept an optional `cwd` parameter for dynamic directory changes during command execution. - Adjusted test case to specify the working directory for asset materialization, ensuring correct file placement. - Added a sleep delay in the test to manage timing issues during execution. - Cleaned up print statements for better clarity in command execution context.
- Moved the `stream_output` function into a private method `_stream_output` for better encapsulation and clarity. - Enhanced `_run_command` method to utilize the new `_stream_output` method for real-time output streaming from subprocesses. - Added logic to ensure the working directory is restored after command execution, improving reliability. - Cleaned up print statements for better context during command execution.
…sterTestContext - Refactored `_run_command` method to utilize a context manager for safe directory changes. - Introduced `_create_subprocess` and `_stream_process_output` methods for better subprocess management and output streaming. - Enhanced `_start_output_threads` method to streamline output capture from subprocesses. - Updated test case to modify the SQL model file directly, improving clarity and functionality. - Removed commented-out code to clean up the test file.
- Added `RunOptions` to the asset materialization call for better control over execution parameters. - Improved logging by printing pre- and post-change dataframes for `intermediate_model_1`. - Cleaned up commented-out code to enhance test clarity and maintainability. - Ensured that the test context initializes the test source correctly for accurate testing.
|
@duragdatadude shouldn't the test_source materialise the table instead of returning a DataFrame? Otherwise it just returns it into std output, right? |
|
@lucargir I don't think I have made any changes to the test source in my PRs. I thought the reason it worked was due to the IO manager that is there (I didn't implement it). If you are talking about the test failures, this is due to something else I believe (trying to better understand SQLMesh's behaviour with backfills and breaking changes - just haven't had a chance to look) |
|
@duragdatadude I was referring to line 173 from the test logs. It mentions that the test table that hast to be materialised before executing the SQLMesh tests, does not exist. |
|
@lucargir Oh, I will look into that. Probably just messed up something while I was moving stuff around to try and get the dagster integration tests working (pretty sure I know what caused it) |
…e test file - Renamed `dagster-materialize` to `mat-test-source` for clarity in the Makefile. - Updated the asset materialization command to select `test_source` instead of `intermediate_model_1`. - Deleted the `test_model_code_change.py` file, which contained extensive commented-out code and was no longer needed. - Improved overall organization and maintainability of the testing structure.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
test_model_code_change.py is killing me. All the other tests are working though.