Commit Graph

242 Commits

Author SHA1 Message Date
Antonio Sánchez
6e4d5d4832 Add IWYU private pragmas to internal headers. 2023-08-21 16:25:22 +00:00
Chip Kerchner
7769eb1b2e Fix problems with recent changes and Tensorflow in Power 2023-07-26 16:24:58 +00:00
Marcus Comstedt
8f927fb52e Altivec: fix compilation with C++20 and higher 2023-07-05 13:14:02 +00:00
Chip Kerchner
3791ac8a1a Fix supportsMMA to obey EIGEN_ALTIVEC_MMA_DYNAMIC_DISPATCH compilation flag and compiler support. 2023-06-28 17:57:21 +00:00
Chip Kerchner
211c5dfc67 Add optional offset parameter to ploadu_partial and pstoreu_partial 2023-06-23 19:53:05 +00:00
Chip Kerchner
b8208b363c Specialized loadColData correctly - fix previous BF16 GEMV MR 2023-05-04 16:38:17 +00:00
Chip Kerchner
fda1373a15 Fix ColMajor BF16 GEMV for when vector is RowMajor 2023-05-03 20:12:50 +00:00
Chip Kerchner
6418ac0285 Unroll F32 to BF16 loop - 1.8X faster conversions for LLVM. Use vector pairs for GCC. 2023-05-01 16:54:16 +00:00
Chip Kerchner
03f646b7e3 New VSX version of BF16 GEMV (Power) - up to 6.7X faster 2023-04-21 17:06:59 +00:00
Chip Kerchner
3f3ce214e6 New BF16 pcast functions and move type casting to TypeCasting.h 2023-04-18 02:38:38 +00:00
Chip Kerchner
1148f0a9ec Add dynamic dispatch to BF16 GEMM (Power) and new VSX version 2023-04-14 22:20:42 +00:00
Rasmus Munk Larsen
df1049ddf4 Small packet math cleanup. 2023-04-04 16:14:32 +00:00
Chip Kerchner
d71ac6a755 Fix recent PowerPC warnings and clang warning 2023-03-15 16:50:46 +00:00
Chip Kerchner
23e1541863 Put deadcode checks back in from previous change. 2023-03-14 00:57:16 +00:00
Chip Kerchner
6c58f0fe1f Revert changes that made BF16 GEMM to cause bad register spillage for LLVM (Power) 2023-03-13 23:36:06 +00:00
Chip Kerchner
9d72412385 Add MMA to BF16 GEMV - 5.0-6.3X faster (for Power) 2023-03-13 19:37:13 +00:00
Rasmus Munk Larsen
ee0ff0ab3a Fix typo in MathFunctions.h 2023-03-13 15:50:40 +00:00
Rasmus Munk Larsen
d6235d76db Clean up generic packetmath specializations for various backends with the help of a macro. 2023-03-10 22:02:23 +00:00
Chip Kerchner
2b513ca2a0 Added partial linear access for LHS & Output - 30% faster for bfloat16 GEMM MMA (Power) 2023-03-02 19:22:43 +00:00
Chip Kerchner
e4598fedbe Fix compiler versions for certain instructions on Power. 2023-02-23 23:24:41 +00:00
Rasmus Munk Larsen
ce62177b5b Vectorize atanh & add a missing definition and unit test for atan. 2023-02-21 03:14:05 +00:00
Chip Kerchner
e797974689 Add and enable Packet int divide for Power10. 2023-02-17 19:04:18 +00:00
Chip Kerchner
4a03409569 Fix problem with array conversions BF16->F32 in Power. 2023-02-13 21:30:45 +00:00
Chip Kerchner
0ecae61568 Disable array BF16 to F32 conversions in Power 2023-02-10 20:06:58 +00:00
Chip Kerchner
fba12e02b3 Fold extra column calculations into an extra MMA accumulator and other bfloat16 MMA GEMM improvements 2023-02-10 17:32:06 +00:00
Chip Kerchner
79cfc74f4d Revert ODR changes and make gemm_extra_cols and gemm_complex_extra_cols EIGEN_ALWAYS_INLINE to avoid external functions. 2023-02-10 17:05:07 +00:00
Alexander Grund
f9659d91f1 Fix ODR violation with gemm_extra_cols on PPC 2023-02-09 22:16:06 +00:00
Chip Kerchner
e71f88abce Change in Power eigen_asserts to eigen_internal_asserts since it is putting unnecessary error checking and assertions without NDEBUG. 2023-02-08 00:57:30 +00:00
Chip Kerchner
4a58f30aa0 Fix pre-POWER8_VECTOR bugs in pcmp_lt and pnegate and reactivate psqrt. 2023-01-31 19:40:24 +00:00
Chip Kerchner
ab8725d947 Turn off vectorize version of rsqrt - doesn't match generic version 2023-01-27 18:28:54 +00:00
Chip Kerchner
6fc9de7d93 Fix slowdown in bfloat16 MMA when rows is not a multiple of 8 or columns is not a multiple of 4. 2023-01-25 18:22:20 +00:00
Sean McBride
d70b4864d9 issue #2581: review and cleanup of compiler version checks 2023-01-17 18:58:34 +00:00
Sergey Fedorov
4d05765345 Altivec fixes for Darwin: do not use unsupported VSX insns 2023-01-12 16:33:33 +00:00
Chip Kerchner
d20fe21ae4 Improve performance for Power10 MMA bfloat16 GEMM 2023-01-06 23:08:37 +00:00
Pedro Caldeira
31ab62d347 Add support for Power10 (AltiVec) MMA instructions for bfloat16. 2022-11-30 23:33:37 +00:00
Chip Kerchner
399ce1ed63 Fix duplicate execution code for Power 8 Altivec in pstore_partial. 2022-11-16 13:41:42 +00:00
Antonio Sanchez
5eacb9e117 Put brackets around unsigned type names. 2022-11-15 09:09:45 -08:00
Antonio Sánchez
37e40dca85 Fix ambiguity in PPC for vec_splats call. 2022-11-14 18:58:16 +00:00
Charles Schlosser
82b152dbe7 Add signbit function 2022-11-04 00:31:20 +00:00
Antonio Sánchez
886aad1361 Disable patan for double on PPC. 2022-10-27 17:56:08 +00:00
Rasmus Munk Larsen
1414a76fa9 Only vectorize atan<double> for Altivec if VSX is available. 2022-10-03 22:06:58 +00:00
Rasmus Munk Larsen
c475228b28 Vectorize atan() for double. 2022-10-01 01:49:30 +00:00
Rasmus Munk Larsen
bd393e15c3 Vectorize acos, asin, and atan for float. 2022-08-29 19:49:33 +00:00
Charles Schlosser
e5af9f87f2 Vectorize pow for integer base / exponent types 2022-08-29 19:23:54 +00:00
Chip Kerchner
9e0afe0f02 Fix non-VSX PowerPC build 2022-08-08 18:18:17 +00:00
Chip Kerchner
84a9d6fac9 Fix use of Packet2d type for non-VSX. 2022-08-03 20:48:13 +00:00
Chip Kerchner
ce60a7be83 Partial Packet support for GEMM real-only (PowerPC). Also fix compilation warnings & errors for some conditions in new API. 2022-08-03 18:15:19 +00:00
Chip Kerchner
84cf3ff18d Add pload_partial, pstore_partial (and unaligned versions), pgather_partial, pscatter_partial, loadPacketPartial and storePacketPartial. 2022-06-27 19:18:00 +00:00
Chip Kerchner
c603275dc9 Better performance for Power10 using more load and store vector pairs for GEMV 2022-06-27 18:11:55 +00:00
Chip Kerchner
4d1c16eab8 Fix tanh and erf to use vectorized version for EIGEN_FAST_MATH in VSX. 2022-06-15 16:06:43 +00:00