refactor: change to TORCH_LIBRARY #823
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
… testcases Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Signed-off-by: abmfy <[email protected]>
Many files in `include/flashinfer/attention/hopper` have their `namespace flashinfer` to `using namespace cute` directly, which is then `using namespace flashinfer`'ed by some files, causing names from `cute` to leak into the global namespace. This is a temporary fix to make compilations work. Signed-off-by: abmfy <[email protected]>
|
Please wait a moment, the CUDA graph issue has been fixed by #822. |
yzh119
added a commit
that referenced
this pull request
Feb 13, 2025
Follow up of #823 , the `CutlassSegmentGEMMSM90` API do not have member `plan_info_vec`
zhyncs
pushed a commit
that referenced
this pull request
Feb 13, 2025
apply #662 again, since we have #823 merged now. --------- Signed-off-by: youkaichao <[email protected]>
yzh119
added a commit
that referenced
this pull request
Feb 13, 2025
Followup of #823 , we should import `from .. import flashinfer_kernels, flashinfer_kernels_sm90` instead of `from .. import _kernels, _kernels_sm90`, otherwise we will be using JIT compilation all the code. Also add some logic to catch "undefined symbol" errors in case the AOT wheel compilation is successful but failed to be loaded.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR updates FlashInfer's C++/CUDA extensions from pybind11 modules to
torch.libraries, which is recommended since PyTorch 2.5.This is mainly implemented in #764. We have investigated that the issue in #820 was not caused by this PR, so we're opening it up again.