Skip to content

Improve invalid backend error #626

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

gordonwatts
Copy link
Collaborator

@gordonwatts gordonwatts commented Jul 15, 2025

Summary

  • improve error message for invalid backend specification - list all backends to help user fix the error.
  • Improve error message when failed to load to include the servicex path where item was added from
  • Added tests to improve coverage

Notes

  • Part of the code was written with codex

https://chatgpt.com/codex/tasks/task_e_6876c3822040832089f2687e0926f63b

@gordonwatts gordonwatts marked this pull request as draft July 15, 2025 21:47
@gordonwatts gordonwatts self-assigned this Jul 15, 2025
@gordonwatts gordonwatts added the enhancement New feature or request label Jul 15, 2025
Copy link

codecov bot commented Jul 15, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.19%. Comparing base (929c2aa) to head (e22cafd).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #626      +/-   ##
==========================================
+ Coverage   96.97%   97.19%   +0.21%     
==========================================
  Files          29       29              
  Lines        1948     1958      +10     
==========================================
+ Hits         1889     1903      +14     
+ Misses         59       55       -4     
Flag Coverage Δ
unittests 97.19% <100.00%> (+0.21%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

@BenGalewsky BenGalewsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this library getting better and more friendly as we feast on our dogfood

@BenGalewsky
Copy link
Contributor

Why is this in draft mode?

@ponyisi
Copy link
Collaborator

ponyisi commented Jul 16, 2025

  • could it have a test?
  • any chance it could replace .servicex with the actual name/path of the config file it reads? It's strange to get messages about .servicex when the file is actually servicex.yaml.

@gordonwatts
Copy link
Collaborator Author

Why is this in draft mode?

Because I was waiting for the tests to finish to make sure there wasn't a failure!!

gordonwatts and others added 2 commits July 22, 2025 12:09
* Improve backend error message

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci
@gordonwatts gordonwatts force-pushed the codex/list-valid-backend-names-in-error-message branch from 5768488 to c73b30b Compare July 22, 2025 16:11
@gordonwatts gordonwatts removed the codex label Jul 22, 2025
@gordonwatts
Copy link
Collaborator Author

Here is what it looks like now:

PS C:\Users\gordo\Code\atlas\object-partitioning> atlas-object-partitioning mc23_13p6TeV:mc23_13p6TeV.601237.PhPy8EG_A14_ttbar_hdamp258p75_allhad.deriv.DAOD_PHYSLITE.e8514_s4369_r16083_p6697 -n 1 --ignore-cache --servicex-name bogus
╭─────────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────────╮
│ C:\Users\gordo\Code\atlas\object-partitioning\src\atlas_object_partitioning\partition.py:36 in main                             │
│                                                                                                                                 │
│   33 │   ),                                                                                                                     │
│   34 ):                                                                                                                         │
│   35 │   """atlas-object-partitioning CLI is working!"""                                                                        │
│ ❱ 36 │   counts = collect_object_counts(                                                                                        │
│   37 │   │   ds_name,                                                                                                           │
│   38 │   │   n_files=n_files,                                                                                                   │
│   39 │   │   servicex_name=servicex_name,                                                                                       │
│                                                                                                                                 │
│ ╭─────────────────────────────────────────────── locals ────────────────────────────────────────────────╮                       │
│ │       ds_name = 'mc23_13p6TeV:mc23_13p6TeV.601237.PhPy8EG_A14_ttbar_hdamp258p75_allhad.deriv.DAOD'+34 │                       │
│ │  ignore_cache = True                                                                                  │                       │
│ │       n_files = 1                                                                                     │                       │
│ │   output_file = 'object_counts.parquet'                                                               │                       │
│ │ servicex_name = 'bogus'                                                                               │                       │
│ ╰───────────────────────────────────────────────────────────────────────────────────────────────────────╯                       │
│                                                                                                                                 │
│ C:\Users\gordo\Code\atlas\object-partitioning\src\atlas_object_partitioning\scan_ds.py:40 in collect_object_counts              │
│                                                                                                                                 │
│   37 │   │   n_files=_nfiles_value(n_files),                                                                                    │
│   38 │   │   title="object_counts",                                                                                             │
│   39 │   )                                                                                                                      │
│ ❱ 40 │   r = deliver(                                                                                                           │
│   41 │   │   spec, backend_name, adaptor=adaptor, ignore_local_cache=ignore_local_cache                                         │
│   42 │   )                                                                                                                      │
│   43                                                                                                                            │
│                                                                                                                                 │
│ ╭────────────────────────────────────────────────────────── locals ───────────────────────────────────────────────────────────╮ │
│ │            adaptor = None                                                                                                   │ │
│ │       backend_name = 'bogus'                                                                                                │ │
│ │            ds_name = 'mc23_13p6TeV:mc23_13p6TeV.601237.PhPy8EG_A14_ttbar_hdamp258p75_allhad.deriv.DAOD'+34                  │ │
│ │ ignore_local_cache = True                                                                                                   │ │
│ │            n_files = 1                                                                                                      │ │
│ │              query = 'FuncADLQueryPHYSLITE'                                                                                 │ │
│ │      servicex_name = 'bogus'                                                                                                │ │
│ │               spec = ServiceXSpec(                                                                                          │ │
│ │                      │   General=General(                                                                                   │ │
│ │                      │   │   Codegen=None,                                                                                  │ │
│ │                      │   │   OutputFormat=<OutputFormatEnum.root_ttree: 'root-ttree'>,                                      │ │
│ │                      │   │   Delivery=<DeliveryEnum.LocalCache: 'LocalCache'>,                                              │ │
│ │                      │   │   OutputDirectory=None,                                                                          │ │
│ │                      │   │   OutFilesetName='servicex_fileset',                                                             │ │
│ │                      │   │   IgnoreLocalCache=False                                                                         │ │
│ │                      │   ),                                                                                                 │ │
│ │                      │   Sample=[                                                                                           │ │
│ │                      │   │   Sample(                                                                                        │ │
│ │                      │   │   │   Name='object_counts',                                                                      │ │
│ │                      │   │   │   Dataset=<servicex.dataset_identifier.RucioDatasetIdentifier object at 0x000001A91AF6D550>, │ │
│ │                      │   │   │   NFiles=1,                                                                                  │ │
│ │                      │   │   │   Query='FuncADLQueryPHYSLITE',                                                              │ │
│ │                      │   │   │   IgnoreLocalCache=True,                                                                     │ │
│ │                      │   │   │   Codegen='atlasr25',                                                                        │ │
│ │                      │   │   │   RucioDID=None,                                                                             │ │
│ │                      │   │   │   XRootDFiles=None                                                                           │ │
│ │                      │   │   )                                                                                              │ │
│ │                      │   ],                                                                                                 │ │
│ │                      │   Definition=None                                                                                    │ │
│ │                      )                                                                                                      │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                                                 │
│ C:\Users\gordo\Code\atlas\object-partitioning\src\atlas_object_partitioning\local_mode.py:136 in deliver                        │
│                                                                                                                                 │
│   133 │   │   │   spec, adaptor=adaptor, ignore_local_cache=ignore_local_cache                                                  │
│   134 │   │   )                                                                                                                 │
│   135 │   else:                                                                                                                 │
│ ❱ 136 │   │   return sx_deliver(                                                                                                │
│   137 │   │   │   spec, servicex_name=servicex_name, ignore_local_cache=ignore_local_cache                                      │
│   138 │   │   )                                                                                                                 │
│   139                                                                                                                           │
│                                                                                                                                 │
│ ╭────────────────────────────────────────────────────────── locals ───────────────────────────────────────────────────────────╮ │
│ │            adaptor = None                                                                                                   │ │
│ │ ignore_local_cache = True                                                                                                   │ │
│ │        run_locally = False                                                                                                  │ │
│ │      servicex_name = 'bogus'                                                                                                │ │
│ │               spec = ServiceXSpec(                                                                                          │ │
│ │                      │   General=General(                                                                                   │ │
│ │                      │   │   Codegen=None,                                                                                  │ │
│ │                      │   │   OutputFormat=<OutputFormatEnum.root_ttree: 'root-ttree'>,                                      │ │
│ │                      │   │   Delivery=<DeliveryEnum.LocalCache: 'LocalCache'>,                                              │ │
│ │                      │   │   OutputDirectory=None,                                                                          │ │
│ │                      │   │   OutFilesetName='servicex_fileset',                                                             │ │
│ │                      │   │   IgnoreLocalCache=False                                                                         │ │
│ │                      │   ),                                                                                                 │ │
│ │                      │   Sample=[                                                                                           │ │
│ │                      │   │   Sample(                                                                                        │ │
│ │                      │   │   │   Name='object_counts',                                                                      │ │
│ │                      │   │   │   Dataset=<servicex.dataset_identifier.RucioDatasetIdentifier object at 0x000001A91AF6D550>, │ │
│ │                      │   │   │   NFiles=1,                                                                                  │ │
│ │                      │   │   │   Query='FuncADLQueryPHYSLITE',                                                              │ │
│ │                      │   │   │   IgnoreLocalCache=True,                                                                     │ │
│ │                      │   │   │   Codegen='atlasr25',                                                                        │ │
│ │                      │   │   │   RucioDID=None,                                                                             │ │
│ │                      │   │   │   XRootDFiles=None                                                                           │ │
│ │                      │   │   )                                                                                              │ │
│ │                      │   ],                                                                                                 │ │
│ │                      │   Definition=None                                                                                    │ │
│ │                      )                                                                                                      │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                                                 │
│ C:\Users\gordo\Code\atlas\object-partitioning\.venv\Lib\site-packages\make_it_sync\func_wrapper.py:63 in wrapped_call           │
│                                                                                                                                 │
│   60 │   else:                                                                                                                  │
│   61 │   │   @wraps(fn)                                                                                                         │
│   62 │   │   def wrapped_call(*args, **kwargs):                                                                                 │
│ ❱ 63 │   │   │   return _sync_version_of_function(fn, *args, **kwargs)                                                          │
│   64 │   │                                                                                                                      │
│   65 │   │   return wrapped_call                                                                                                │
│   66                                                                                                                            │
│                                                                                                                                 │
│ ╭────────────────────────────────────────────────────── locals ───────────────────────────────────────────────────────╮         │
│ │   args = (                                                                                                          │         │
│ │          │   ServiceXSpec(                                                                                          │         │
│ │          │   │   General=General(                                                                                   │         │
│ │          │   │   │   Codegen=None,                                                                                  │         │
│ │          │   │   │   OutputFormat=<OutputFormatEnum.root_ttree: 'root-ttree'>,                                      │         │
│ │          │   │   │   Delivery=<DeliveryEnum.LocalCache: 'LocalCache'>,                                              │         │
│ │          │   │   │   OutputDirectory=None,                                                                          │         │
│ │          │   │   │   OutFilesetName='servicex_fileset',                                                             │         │
│ │          │   │   │   IgnoreLocalCache=False                                                                         │         │
│ │          │   │   ),                                                                                                 │         │
│ │          │   │   Sample=[                                                                                           │         │
│ │          │   │   │   Sample(                                                                                        │         │
│ │          │   │   │   │   Name='object_counts',                                                                      │         │
│ │          │   │   │   │   Dataset=<servicex.dataset_identifier.RucioDatasetIdentifier object at 0x000001A91AF6D550>, │         │
│ │          │   │   │   │   NFiles=1,                                                                                  │         │
│ │          │   │   │   │   Query='FuncADLQueryPHYSLITE',                                                              │         │
│ │          │   │   │   │   IgnoreLocalCache=True,                                                                     │         │
│ │          │   │   │   │   Codegen='atlasr25',                                                                        │         │
│ │          │   │   │   │   RucioDID=None,                                                                             │         │
│ │          │   │   │   │   XRootDFiles=None                                                                           │         │
│ │          │   │   │   )                                                                                              │         │
│ │          │   │   ],                                                                                                 │         │
│ │          │   │   Definition=None                                                                                    │         │
│ │          │   ),                                                                                                     │         │
│ │          )                                                                                                          │         │
│ │ kwargs = {'servicex_name': 'bogus', 'ignore_local_cache': True}                                                     │         │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯         │
│                                                                                                                                 │
│ C:\Users\gordo\Code\atlas\object-partitioning\.venv\Lib\site-packages\make_it_sync\func_wrapper.py:14 in                        │
│ _sync_version_of_function                                                                                                       │
│                                                                                                                                 │
│   11 │   if not loop.is_running():                                                                                              │
│   12 │   │   # Call the function directly                                                                                       │
│   13 │   │   r = fn(*args, **kwargs)                                                                                            │
│ ❱ 14 │   │   return loop.run_until_complete(r)                                                                                  │
│   15 │   else:                                                                                                                  │
│   16 │   │   def get_data_wrapper(*args, **kwargs):                                                                             │
│   17 │   │   │   # New thread - get the loop.                                                                                   │
│                                                                                                                                 │
│ ╭────────────────────────────────────────────────────── locals ───────────────────────────────────────────────────────╮         │
│ │   args = (                                                                                                          │         │
│ │          │   ServiceXSpec(                                                                                          │         │
│ │          │   │   General=General(                                                                                   │         │
│ │          │   │   │   Codegen=None,                                                                                  │         │
│ │          │   │   │   OutputFormat=<OutputFormatEnum.root_ttree: 'root-ttree'>,                                      │         │
│ │          │   │   │   Delivery=<DeliveryEnum.LocalCache: 'LocalCache'>,                                              │         │
│ │          │   │   │   OutputDirectory=None,                                                                          │         │
│ │          │   │   │   OutFilesetName='servicex_fileset',                                                             │         │
│ │          │   │   │   IgnoreLocalCache=False                                                                         │         │
│ │          │   │   ),                                                                                                 │         │
│ │          │   │   Sample=[                                                                                           │         │
│ │          │   │   │   Sample(                                                                                        │         │
│ │          │   │   │   │   Name='object_counts',                                                                      │         │
│ │          │   │   │   │   Dataset=<servicex.dataset_identifier.RucioDatasetIdentifier object at 0x000001A91AF6D550>, │         │
│ │          │   │   │   │   NFiles=1,                                                                                  │         │
│ │          │   │   │   │   Query='FuncADLQueryPHYSLITE',                                                              │         │
│ │          │   │   │   │   IgnoreLocalCache=True,                                                                     │         │
│ │          │   │   │   │   Codegen='atlasr25',                                                                        │         │
│ │          │   │   │   │   RucioDID=None,                                                                             │         │
│ │          │   │   │   │   XRootDFiles=None                                                                           │         │
│ │          │   │   │   )                                                                                              │         │
│ │          │   │   ],                                                                                                 │         │
│ │          │   │   Definition=None                                                                                    │         │
│ │          │   ),                                                                                                     │         │
│ │          )                                                                                                          │         │
│ │ kwargs = {'servicex_name': 'bogus', 'ignore_local_cache': True}                                                     │         │
│ │   loop = <ProactorEventLoop running=False closed=False debug=False>                                                 │         │
│ │      r = <coroutine object deliver_async at 0x000001A91AF6F220>                                                     │         │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯         │
│                                                                                                                                 │
│ C:\Users\gordo\AppData\Local\Programs\Python\Python313\Lib\asyncio\base_events.py:725 in run_until_complete                     │
│                                                                                                                                 │
│    722 │   │   if not future.done():                                                                                            │
│    723 │   │   │   raise RuntimeError('Event loop stopped before Future completed.')                                            │
│    724 │   │                                                                                                                    │
│ ❱  725 │   │   return future.result()                                                                                           │
│    726 │                                                                                                                        │
│    727 │   def stop(self):                                                                                                      │
│    728 │   │   """Stop running the event loop.                                                                                  │
│                                                                                                                                 │
│ ╭────────────────────────────────────────────────────────── locals ───────────────────────────────────────────────────────────╮ │
│ │   future = <Task finished name='Task-1' coro=<deliver_async() done, defined at                                              │ │
│ │            C:\Users\gordo\Code\atlas\object-partitioning\.venv\Lib\site-packages\servicex\servicex_client.py:236>           │ │
│ │            exception=ValueError('Backend bogus not defined in C:\\Users\\gordo\\servicex.yaml file. Valid backend names:    │ │
│ │            servicex, servicex-release-testing-3, servicex-release-prod')>                                                   │ │
│ │ new_task = True                                                                                                             │ │
│ │     self = <ProactorEventLoop running=False closed=False debug=False>                                                       │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                                                 │
│ C:\Users\gordo\Code\atlas\object-partitioning\.venv\Lib\site-packages\servicex\servicex_client.py:279 in deliver_async          │
│                                                                                                                                 │
│   276 │   │   for sample in config.Sample:                                                                                      │
│   277 │   │   │   sample.IgnoreLocalCache = True                                                                                │
│   278 │                                                                                                                         │
│ ❱ 279 │   datasets = await _build_datasets(                                                                                     │
│   280 │   │   config, config_path, servicex_name, fail_if_incomplete                                                            │
│   281 │   )                                                                                                                     │
│   282                                                                                                                           │
│                                                                                                                                 │
│ ╭────────────────────────────────────────────────────────── locals ───────────────────────────────────────────────────────────╮ │
│ │        concurrency = 10                                                                                                     │ │
│ │             config = ServiceXSpec(                                                                                          │ │
│ │                      │   General=General(                                                                                   │ │
│ │                      │   │   Codegen=None,                                                                                  │ │
│ │                      │   │   OutputFormat=<OutputFormatEnum.root_ttree: 'root-ttree'>,                                      │ │
│ │                      │   │   Delivery=<DeliveryEnum.LocalCache: 'LocalCache'>,                                              │ │
│ │                      │   │   OutputDirectory=None,                                                                          │ │
│ │                      │   │   OutFilesetName='servicex_fileset',                                                             │ │
│ │                      │   │   IgnoreLocalCache=False                                                                         │ │
│ │                      │   ),                                                                                                 │ │
│ │                      │   Sample=[                                                                                           │ │
│ │                      │   │   Sample(                                                                                        │ │
│ │                      │   │   │   Name='object_counts',                                                                      │ │
│ │                      │   │   │   Dataset=<servicex.dataset_identifier.RucioDatasetIdentifier object at 0x000001A91AF6D550>, │ │
│ │                      │   │   │   NFiles=1,                                                                                  │ │
│ │                      │   │   │   Query='FuncADLQueryPHYSLITE',                                                              │ │
│ │                      │   │   │   IgnoreLocalCache=True,                                                                     │ │
│ │                      │   │   │   Codegen='atlasr25',                                                                        │ │
│ │                      │   │   │   RucioDID=None,                                                                             │ │
│ │                      │   │   │   XRootDFiles=None                                                                           │ │
│ │                      │   │   )                                                                                              │ │
│ │                      │   ],                                                                                                 │ │
│ │                      │   Definition=None                                                                                    │ │
│ │                      )                                                                                                      │ │
│ │        config_path = None                                                                                                   │ │
│ │ fail_if_incomplete = True                                                                                                   │ │
│ │ ignore_local_cache = True                                                                                                   │ │
│ │       progress_bar = <ProgressBarFormat.expanded: 'expanded'>                                                               │ │
│ │  return_exceptions = True                                                                                                   │ │
│ │             sample = Sample(                                                                                                │ │
│ │                      │   Name='object_counts',                                                                              │ │
│ │                      │   Dataset=<servicex.dataset_identifier.RucioDatasetIdentifier object at 0x000001A91AF6D550>,         │ │
│ │                      │   NFiles=1,                                                                                          │ │
│ │                      │   Query='FuncADLQueryPHYSLITE',                                                                      │ │
│ │                      │   IgnoreLocalCache=True,                                                                             │ │
│ │                      │   Codegen='atlasr25',                                                                                │ │
│ │                      │   RucioDID=None,                                                                                     │ │
│ │                      │   XRootDFiles=None                                                                                   │ │
│ │                      )                                                                                                      │ │
│ │      servicex_name = 'bogus'                                                                                                │ │
│ │               spec = ServiceXSpec(                                                                                          │ │
│ │                      │   General=General(                                                                                   │ │
│ │                      │   │   Codegen=None,                                                                                  │ │
│ │                      │   │   OutputFormat=<OutputFormatEnum.root_ttree: 'root-ttree'>,                                      │ │
│ │                      │   │   Delivery=<DeliveryEnum.LocalCache: 'LocalCache'>,                                              │ │
│ │                      │   │   OutputDirectory=None,                                                                          │ │
│ │                      │   │   OutFilesetName='servicex_fileset',                                                             │ │
│ │                      │   │   IgnoreLocalCache=False                                                                         │ │
│ │                      │   ),                                                                                                 │ │
│ │                      │   Sample=[                                                                                           │ │
│ │                      │   │   Sample(                                                                                        │ │
│ │                      │   │   │   Name='object_counts',                                                                      │ │
│ │                      │   │   │   Dataset=<servicex.dataset_identifier.RucioDatasetIdentifier object at 0x000001A91AF6D550>, │ │
│ │                      │   │   │   NFiles=1,                                                                                  │ │
│ │                      │   │   │   Query='FuncADLQueryPHYSLITE',                                                              │ │
│ │                      │   │   │   IgnoreLocalCache=True,                                                                     │ │
│ │                      │   │   │   Codegen='atlasr25',                                                                        │ │
│ │                      │   │   │   RucioDID=None,                                                                             │ │
│ │                      │   │   │   XRootDFiles=None                                                                           │ │
│ │                      │   │   )                                                                                              │ │
│ │                      │   ],                                                                                                 │ │
│ │                      │   Definition=None                                                                                    │ │
│ │                      )                                                                                                      │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                                                 │
│ C:\Users\gordo\Code\atlas\object-partitioning\.venv\Lib\site-packages\servicex\servicex_client.py:181 in _build_datasets        │
│                                                                                                                                 │
│   178 │   │   elif isinstance(_sample.Query, Query):                                                                            │
│   179 │   │   │   return _sample.Query.codegen                                                                                  │
│   180 │                                                                                                                         │
│ ❱ 181 │   sx = ServiceXClient(backend=servicex_name, config_path=config_path)                                                   │
│   182 │   title_length_limit = await sx.servicex.get_servicex_sample_title_limit()                                              │
│   183 │   datasets = []                                                                                                         │
│   184 │   for sample in config.Sample:                                                                                          │
│                                                                                                                                 │
│ ╭────────────────────────────────────────────────────────── locals ───────────────────────────────────────────────────────────╮ │
│ │             config = ServiceXSpec(                                                                                          │ │
│ │                      │   General=General(                                                                                   │ │
│ │                      │   │   Codegen=None,                                                                                  │ │
│ │                      │   │   OutputFormat=<OutputFormatEnum.root_ttree: 'root-ttree'>,                                      │ │
│ │                      │   │   Delivery=<DeliveryEnum.LocalCache: 'LocalCache'>,                                              │ │
│ │                      │   │   OutputDirectory=None,                                                                          │ │
│ │                      │   │   OutFilesetName='servicex_fileset',                                                             │ │
│ │                      │   │   IgnoreLocalCache=False                                                                         │ │
│ │                      │   ),                                                                                                 │ │
│ │                      │   Sample=[                                                                                           │ │
│ │                      │   │   Sample(                                                                                        │ │
│ │                      │   │   │   Name='object_counts',                                                                      │ │
│ │                      │   │   │   Dataset=<servicex.dataset_identifier.RucioDatasetIdentifier object at 0x000001A91AF6D550>, │ │
│ │                      │   │   │   NFiles=1,                                                                                  │ │
│ │                      │   │   │   Query='FuncADLQueryPHYSLITE',                                                              │ │
│ │                      │   │   │   IgnoreLocalCache=True,                                                                     │ │
│ │                      │   │   │   Codegen='atlasr25',                                                                        │ │
│ │                      │   │   │   RucioDID=None,                                                                             │ │
│ │                      │   │   │   XRootDFiles=None                                                                           │ │
│ │                      │   │   )                                                                                              │ │
│ │                      │   ],                                                                                                 │ │
│ │                      │   Definition=None                                                                                    │ │
│ │                      )                                                                                                      │ │
│ │        config_path = None                                                                                                   │ │
│ │ fail_if_incomplete = True                                                                                                   │ │
│ │      servicex_name = 'bogus'                                                                                                │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                                                 │
│ C:\Users\gordo\Code\atlas\object-partitioning\.venv\Lib\site-packages\servicex\servicex_client.py:348 in __init__               │
│                                                                                                                                 │
│   345 │   │   │   if backend not in self.endpoints:                                                                             │
│   346 │   │   │   │   valid_backends = ", ".join(self.endpoints.keys())                                                         │
│   347 │   │   │   │   cfg_file = self.config.config_file or ".servicex"                                                         │
│ ❱ 348 │   │   │   │   raise ValueError(                                                                                         │
│   349 │   │   │   │   │   f"Backend {backend} not defined in {cfg_file} file. "                                                 │
│   350 │   │   │   │   │   f"Valid backend names: {valid_backends}"                                                              │
│   351 │   │   │   │   )                                                                                                         │
│                                                                                                                                 │
│ ╭──────────────────────────────────────── locals ─────────────────────────────────────────╮                                     │
│ │        backend = 'bogus'                                                                │                                     │
│ │       cfg_file = 'C:\\Users\\gordo\\servicex.yaml'                                      │                                     │
│ │    config_path = None                                                                   │                                     │
│ │           self = <servicex.servicex_client.ServiceXClient object at 0x000001A91AF6F380> │                                     │
│ │            url = None                                                                   │                                     │
│ │ valid_backends = 'servicex, servicex-release-testing-3, servicex-release-prod'          │                                     │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────╯                                     │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: Backend bogus not defined in C:\Users\gordo\servicex.yaml file. Valid backend names: servicex,
servicex-release-testing-3, servicex-release-prod

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves error handling in the ServiceX client by enhancing error messages for invalid backend specifications and configuration loading issues. The changes make error messages more helpful by providing specific file paths and listing available backends to help users fix configuration problems.

  • Enhanced error messages for invalid backend names to include the config file path and list valid options
  • Added tracking of configuration file paths through the Configuration class
  • Added comprehensive test coverage for the new error handling functionality

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
servicex/configuration.py Modified to track and return config file paths, updated _add_from_path method to return both config and file path
servicex/servicex_client.py Enhanced error message for invalid backends to include config file path and list of valid backend names
tests/test_servicex_client.py Added test to verify error message includes filename when invalid backend is specified
tests/test_config.py Added parameterized test to verify config can be read from default files (servicex.yaml and .servicex)

@gordonwatts gordonwatts marked this pull request as ready for review July 22, 2025 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants