Commit Graph

1435 Commits

Author SHA1 Message Date
Chip Kerchner
4f14da11d9 Add basic support for packetmath for BF16 RVV
libeigen/eigen!2093
2025-12-16 14:25:46 -05:00
Yu You
a7209fad70 GemmKernel: Define static constexpr member variables out-of-class for C++14 compatibility
libeigen/eigen!2091
2025-12-14 01:00:12 +00:00
Chip Kerchner
26fe567dd2 Fix FP16 for RVV so that it will compile for gcc
libeigen/eigen!2090
2025-12-10 08:42:26 -05:00
Chip Kerchner
8cdc0fa67d Fix naming of predux_half for RVV when LMUL > 1
libeigen/eigen!2087
2025-12-05 13:50:16 -05:00
Rasmus Munk Larsen
75bcd155c4 Vectorize tan(x)
libeigen/eigen!2086

Co-authored-by: Rasmus Munk Larsen <rmlarsen@google.com>
2025-12-02 21:53:10 +00:00
Rasmus Munk Larsen
db90c4939c Add a ptanh_float implementation that is accurate to 1 ULP
libeigen/eigen!2082

Co-authored-by: Rasmus Munk Larsen <rmlarsen@google.com>
2025-11-26 00:17:12 +00:00
Chip Kerchner
49623d0c4e This patch adds support for RISCV's vector extension RVV1.0.
libeigen/eigen!2030
2025-11-20 16:28:07 +00:00
Rasmus Munk Larsen
8eb6551a8a Add support for complex numbers in the generic clang backend
libeigen/eigen!2078

Co-authored-by: Rasmus Munk Larsen <rmlarsen@google.com>
2025-11-20 00:26:37 +00:00
Rasmus Munk Larsen
a7674b70d3 Improve packet op test coverage for IEEE special values.
libeigen/eigen!2075

Co-authored-by: Rasmus Munk Larsen <rmlarsen@google.com>
2025-11-12 22:19:50 +00:00
Charles Schlosser
8ae3b1aaa5 Fix loongarch unsigned pabsdiff
libeigen/eigen!2071
2025-11-09 19:19:43 +00:00
Rasmus Munk Larsen
23a5482fc0 Misc. packet math cleanups.
libeigen/eigen!2070

Co-authored-by: Rasmus Munk Larsen <rmlarsen@google.com>
2025-11-08 21:57:20 +00:00
Rasmus Munk Larsen
ffcd7bdbd6 Avoid breaking the build on older compilers.
See merge request libeigen/eigen!2068

Co-authored-by: Rasmus Munk Larsen <rmlarsen@google.com>
2025-11-07 21:25:09 +00:00
Antonio Sánchez
da867c31c9 Fix defines in AVX512 custom TRSM kernel.
<!-- 
Thanks for contributing a merge request!

We recommend that first-time contributors read our [contribution guidelines](https://eigen.tuxfamily.org/index.php?title=Contributing_to_Eigen).

Before submitting the MR, please complete the following checks:
- Create one PR per feature or bugfix,
- Run the test suite to verify your changes.
  See our [test guidelines](https://eigen.tuxfamily.org/index.php?title=Tests).
- Add tests to cover the bug addressed or any new feature.
- Document new features.  If it is a substantial change, add it to the [Changelog](https://gitlab.com/libeigen/eigen/-/blob/master/CHANGELOG.md).
- Leave the following box checked when submitting: `Allow commits from members who can merge to the target branch`.
  This allows us to rebase and merge your change.

Note that we are a team of volunteers; we appreciate your patience during the review process.
-->

### Description
<!--Please explain your changes.-->

Broken at head by !2063.

See merge request libeigen/eigen!2066
2025-11-07 20:14:24 +00:00
Rasmus Munk Larsen
8a9bfb72d7 Rename preduce_half for HVX. 2025-11-07 16:52:07 +00:00
Antonio Sánchez
ed989c7504 Enable generic clang backend tests.
<!-- 
Thanks for contributing a merge request!

We recommend that first-time contributors read our [contribution guidelines](https://eigen.tuxfamily.org/index.php?title=Contributing_to_Eigen).

Before submitting the MR, please complete the following checks:
- Create one PR per feature or bugfix,
- Run the test suite to verify your changes.
  See our [test guidelines](https://eigen.tuxfamily.org/index.php?title=Tests).
- Add tests to cover the bug addressed or any new feature.
- Document new features.  If it is a substantial change, add it to the [Changelog](https://gitlab.com/libeigen/eigen/-/blob/master/CHANGELOG.md).
- Leave the following box checked when submitting: `Allow commits from members who can merge to the target branch`.
  This allows us to rebase and merge your change.

Note that we are a team of volunteers; we appreciate your patience during the review process.
-->

### Description
<!--Please explain your changes.-->

Enable generic clang backend tests.

Added an AVX512 job using the generic clang backend.

Also fixed up some guards in the custom AVX512 gemm/trsm kernels so they don't
start defining things if they aren't used.

See merge request libeigen/eigen!2063
2025-11-07 01:37:12 +00:00
Rasmus Munk Larsen
fecfa7f27e Fixes to make generic backend build with AVX512
See merge request libeigen/eigen!2064

Co-authored-by: Rasmus Munk Larsen <rmlarsen@google.com>
2025-11-06 22:50:32 +00:00
Rasmus Munk Larsen
ec93a6d098 Add a generic Eigen backend based on clang vector extensions
The goal of this MR is to implement a generic SIMD backend (packet ops) for Eigen that uses clang vector extensions instead of platform-dependent intrinsics. Ideally, this should make it possible to build Eigen and achieve reasonable speed on any platform that has a recent clang compiler, without having to write any inline assembly or intrinsics.

Caveats:

* The current implementation is a proof of concept and supports vectorization for float, double, int32_t, and int64_t using fixed-size 512-bit vectors (a somewhat arbitrary choice). I have not done much to tune this for speed yet.
* For now, there is no way to enable this other than setting -DEIGEN_VECTORIZE_GENERIC on the command line.
* This only compiles with newer versions of clang. I have tested that it compiles and all tests pass with clang 19.1.7.

https://clang.llvm.org/docs/LanguageExtensions.html#vectors-and-extended-vectors

Closes #2998 and #2997

See merge request libeigen/eigen!2051

Co-authored-by: Rasmus Munk Larsen <rmlarsen@google.com>
Co-authored-by: Antonio Sánchez <cantonios@google.com>
2025-11-06 21:52:19 +00:00
Rasmus Munk Larsen
b6fcddccfc Get rid of pblend packet op.
There was only a single code path left in TensorEvaluator using pblend. We can replace that with a call to the more general TernarySelectOp and get rid of pblend entirely from Core.

Closes #2998

See merge request libeigen/eigen!2056

Co-authored-by: Rasmus Munk Larsen <rmlarsen@google.com>
2025-11-03 23:27:50 +00:00
Rasmus Munk Larsen
ed9a0e59ba Fix more bugs in !2052
Fixes #2998

Closes #2998

See merge request libeigen/eigen!2057

Co-authored-by: Rasmus Munk Larsen <rmlarsen@google.com>
2025-11-03 20:26:17 +00:00
Rasmus Munk Larsen
bfdbc031c2 Fixes #2998. 2025-11-02 23:58:16 +00:00
Rasmus Munk Larsen
8716f109e4 Implement assume_aligned using the standard API
This implements `Eigen::internal::assume_aligned` to match the API for C++20 standard as best as possible using either `std::assume_aligned` or `__builtin_assume_aligned` if available. If neither is available, the function is a no-op.

The override macro `EIGEN_ASSUME_ALIGNED` was changed to a `EIGEN_DONT_ASSUME_ALIGNED`, which now forces the function to be a no-op.

See merge request libeigen/eigen!2052
2025-11-01 12:04:19 +00:00
Rasmus Munk Larsen
ce70a507c0 Enable more generic packet ops for double.
See merge request libeigen/eigen!2050
2025-10-30 19:14:43 +00:00
Rasmus Munk Larsen
ece9a4c0b6 Always vectorize abs2() for non-complex types.
For several packet types, `abs2` was not vectorized even if it only requires `pmul`. Get rid of the confusing and redundant `HasAbs2` enum and instead check `HasMul` in addition to making sure the scalar type is not complex.

See merge request libeigen/eigen!2048
2025-10-30 02:42:56 +00:00
Antonio Sánchez
8e60d4173c Support AVX for i686.
<!-- 
Thanks for contributing a merge request!

We recommend that first-time contributors read our [contribution guidelines](https://eigen.tuxfamily.org/index.php?title=Contributing_to_Eigen).

Before submitting the MR, please complete the following checks:
- Create one PR per feature or bugfix,
- Run the test suite to verify your changes.
  See our [test guidelines](https://eigen.tuxfamily.org/index.php?title=Tests).
- Add tests to cover the bug addressed or any new feature.
- Document new features.  If it is a substantial change, add it to the [Changelog](https://gitlab.com/libeigen/eigen/-/blob/master/CHANGELOG.md).
- Leave the following box checked when submitting: `Allow commits from members who can merge to the target branch`.
  This allows us to rebase and merge your change.

Note that we are a team of volunteers; we appreciate your patience during the review process.
-->

### Description
<!--Please explain your changes.-->

Support AVX for i686.

There was an existing work-around for windows.  Added the more generic
architecture comparison to also apply for linux.

### Reference issue
<!--
You can link to a specific issue using the gitlab syntax #<issue number>. 
If the MR fixes an issue, write "Fixes #<issue number>" to have the issue automatically closed on merge.
-->

Fixes #2991.

Closes #2991

See merge request libeigen/eigen!2037
2025-10-20 21:47:42 +00:00
Rasmus Munk Larsen
565db1c603 Optimize ApplyOnTheRight for Jacobi rotations when FMA is available 2025-10-12 00:04:19 +00:00
Charles Schlosser
5996176b88 Fix alignment bug in avx pcast<Packet4l, Packet4d> 2025-10-09 02:50:42 +00:00
Laurenz
4bd382df56 Fix SSE PacketMath Compilation Error on QNX 2025-10-08 17:13:16 +00:00
Rasmus Munk Larsen
2d170aea11 Define pcmp_le generically in terms of pcmp_eq and pcmp_lt. 2025-09-23 14:34:57 +00:00
Sergiu Deitsch
ea869e183b Add missing bool SSE2 PacketMath comparison 2025-09-22 21:28:45 +02:00
Antonio Sánchez
7f0cb638c5 Specialize numext::madd for half/bfloat16. 2025-08-29 18:11:25 +00:00
Antonio Sánchez
2e8cc042a1 Replace calls to numext::fma with numext:madd. 2025-08-28 21:40:19 +00:00
Aleksei Nikiforov
c487a4fe9e Clean up most of testsuite on s390x 2025-08-15 20:04:25 +00:00
Cheng Wang
1c0048a08c Fix inconsistency between ptrue and pcmp_* in HVX 2025-08-09 19:32:30 +00:00
Rasmus Munk Larsen
4be7e6b4e0 Fix pcmp_* for HVX to comply with the new definition of true = Scalar(1) 2025-08-04 20:56:24 +00:00
Rasmus Munk Larsen
97c7cc6200 Explicitly use the packet trait HasPow to control whether Pow is vectorized. 2025-07-18 21:51:42 +00:00
Charles Schlosser
302fc46bc3 arm packet alignment requirements and aligned loads/stores 2025-07-15 23:49:04 +00:00
Antonio Sánchez
26616fe5b8 Fix VSX packetmath psin and pcast tests. 2025-06-27 04:08:20 +00:00
Antonio Sánchez
a395ee162d Fix a collection of random failures encountered when testing with Bazel. 2025-06-26 16:58:24 +00:00
Antonio Sánchez
db8bd5b825 Modify pselect and various masks to use Scalar(1) for true. 2025-06-20 22:40:46 +00:00
Charles Schlosser
bcce88c99e Faster emulated half comparisons 2025-06-17 17:05:58 +00:00
Charles Schlosser
d228bcdf8f Fix neon compilation bug 2025-06-10 21:52:01 +00:00
Charles Schlosser
994f3d107a Fix neon packet math tests, add missing neon intrinsics 2025-06-09 17:13:31 +00:00
Charles Schlosser
21e89b930c Enable default behavior for pmin<PropagateFast>, predux_min, etc 2025-06-02 17:23:37 +00:00
Charles Schlosser
4fdf87bbf5 clean up intel packet reductions 2025-05-30 19:18:07 +00:00
Tyler Veness
619be0deb6 Replace instances of EIGEN_NOEXCEPT macros 2025-04-22 00:58:47 +00:00
Rasmus Munk Larsen
66d8111ac1 Use a more conservative method to detect non-finite inputs to cbrt. 2025-04-21 20:59:46 +00:00
Tyler Veness
d6689a15d7 Replace instances of EIGEN_CONSTEXPR macro 2025-04-18 08:27:52 -07:00
Rasmus Munk Larsen
33f5f59614 Vectorize cbrt for float and double. 2025-04-17 23:31:20 +00:00
Charles Schlosser
28c3b26d53 masked load/store framework 2025-04-12 00:31:10 +00:00
Antonio Sánchez
02d9e1138a Add missing pmadd for Packet16bf. 2025-03-31 04:17:17 +00:00