Transform http-parser and http-writer with the new Multidict-CAPI (Currently a draft / Vizonex's Playground / Sandbox) #11320
Conversation
for more information, see https://pre-commit.ci
…m/Vizonex/aiohttp into Vizonex-playground-multidict-capi
psf-chronographer
bot
added
the
bot:chronographer:provided
CodSpeed Performance ReportMerging #11320 will improve performances by 37.39%Comparing Summary
Benchmarks breakdown
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #11320 +/- ##
=======================================
Coverage 98.76% 98.76%
=======================================
Files 129 129
Lines 43374 43375 +1
Branches 2323 2323
=======================================
+ Hits 42837 42838 +1
Misses 383 383
Partials 154 154
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
This was the benchmark I was looking for... |
|
Speed up looks real 👍 |
@bdraco Thanks :) , Seems my hard work on multidict paid off. Even with writing a seperate tool that used libclang to make the |
…est Error Protocol Message since Cython is Now Strict about it which is a good thing.
…oxy to more arguments instead of object type
What do these changes do?
This currently is my playground of new things that is planned for multidict 6.7 which I was involved in helping write to hopefully optimize the _http_parser and _http_writer. I loved working with the llhttp http parser myself personally and seeing my work start to merge with it is nothing short of extortionary. This is not only an historical moment but could be final optimizations and benchmarks we were looking for. I wanted to spend my summer doing ethical things with my time and it seems I have hit my goals. Never thought this would be the endgame.
For right now I have hacked in a cython file that I currently am autogenerating with libclang. I left a copy in there so you can use it and not have to download the clang.dll stuff. I saved you from needing that.
Currently waiting on 6.7 to come out and with my cython portions still requiring review this will by no means take a while. I have written some notes that label and warn that these could be all held subject to change.
Edit: I have pointed everything to the multidict fork so no need to do any crazy setups.
vs-code extensions like cyright should be able to properly display what exists. If none of the cpython or libc data exists pip installing cython should do the trick.
Know that my parts and changes will not compile yet, I am waiting for multidict 6.7 to release first however but I am certain that I did the _http_writer parts correctly.
To simplify this excitement for those not wishing to read the importance of this pull request it is simply me waiting for the Multidict-CAPI and having http_writer and http_parser utilize it in cython as the final major performance optimizations.
Are there changes in behavior for the user?
It mainly involves internal refactoring however users should see at least some performance boosts when were done here. We should be able to remain on par with curl-cffi in speed if everything is done correctly, this is my agenda and goal of this pull request.
Is it a substantial burden for the maintainers to support this?
With all the steps I explained above this should not be a burden as this has all currently been carefully crafted and planned for the final product.
Related issue number
Checklist
CONTRIBUTORS.txtCHANGES/foldername it
<issue_or_pr_num>.<type>.rst(e.g.588.bugfix.rst)if you don't have an issue number, change it to the pull request
number after creating the PR
.bugfix: A bug fix for something the maintainers deemed animproper undesired behavior that got corrected to match
pre-agreed expectations.
.feature: A new behavior, public APIs. That sort of stuff..deprecation: A declaration of future API removals and breakingchanges in behavior.
.breaking: When something public is removed in a breaking way.Could be deprecated in an earlier release.
.doc: Notable updates to the documentation structure or buildprocess.
.packaging: Notes for downstreams about unobvious side effectsand tooling. Changes in the test invocation considerations and
runtime assumptions.
.contrib: Stuff that affects the contributor experience. e.g.Running tests, building the docs, setting up the development
environment.
.misc: Changes that are hard to assign to any of the abovecategories.
Make sure to use full sentences with correct case and punctuation,
for example:
Use the past tense or the present tense a non-imperative mood,
referring to what's changed compared to the last released version
of this project.