224 Commits

Author	SHA1	Message	Date
Chip Kerchner	2b513ca2a0	Added partial linear access for LHS & Output - 30% faster for bfloat16 GEMM MMA (Power)	2023-03-02 19:22:43 +00:00
Chip Kerchner	e4598fedbe	Fix compiler versions for certain instructions on Power.	2023-02-23 23:24:41 +00:00
Rasmus Munk Larsen	ce62177b5b	Vectorize atanh & add a missing definition and unit test for atan.	2023-02-21 03:14:05 +00:00
Chip Kerchner	e797974689	Add and enable Packet int divide for Power10.	2023-02-17 19:04:18 +00:00
Chip Kerchner	4a03409569	Fix problem with array conversions BF16->F32 in Power.	2023-02-13 21:30:45 +00:00
Chip Kerchner	0ecae61568	Disable array BF16 to F32 conversions in Power	2023-02-10 20:06:58 +00:00
Chip Kerchner	fba12e02b3	Fold extra column calculations into an extra MMA accumulator and other bfloat16 MMA GEMM improvements	2023-02-10 17:32:06 +00:00
Chip Kerchner	79cfc74f4d	Revert ODR changes and make gemm_extra_cols and gemm_complex_extra_cols EIGEN_ALWAYS_INLINE to avoid external functions.	2023-02-10 17:05:07 +00:00
Alexander Grund	f9659d91f1	Fix ODR violation with gemm_extra_cols on PPC	2023-02-09 22:16:06 +00:00
Chip Kerchner	e71f88abce	Change in Power eigen_asserts to eigen_internal_asserts since it is putting unnecessary error checking and assertions without NDEBUG.	2023-02-08 00:57:30 +00:00
Chip Kerchner	4a58f30aa0	Fix pre-POWER8_VECTOR bugs in pcmp_lt and pnegate and reactivate psqrt.	2023-01-31 19:40:24 +00:00
Chip Kerchner	ab8725d947	Turn off vectorize version of rsqrt - doesn't match generic version	2023-01-27 18:28:54 +00:00
Chip Kerchner	6fc9de7d93	Fix slowdown in bfloat16 MMA when rows is not a multiple of 8 or columns is not a multiple of 4.	2023-01-25 18:22:20 +00:00
Sean McBride	d70b4864d9	issue #2581: review and cleanup of compiler version checks	2023-01-17 18:58:34 +00:00
Sergey Fedorov	4d05765345	Altivec fixes for Darwin: do not use unsupported VSX insns	2023-01-12 16:33:33 +00:00
Chip Kerchner	d20fe21ae4	Improve performance for Power10 MMA bfloat16 GEMM	2023-01-06 23:08:37 +00:00
Pedro Caldeira	31ab62d347	Add support for Power10 (AltiVec) MMA instructions for bfloat16.	2022-11-30 23:33:37 +00:00
Chip Kerchner	399ce1ed63	Fix duplicate execution code for Power 8 Altivec in pstore_partial.	2022-11-16 13:41:42 +00:00
Antonio Sanchez	5eacb9e117	Put brackets around unsigned type names.	2022-11-15 09:09:45 -08:00
Antonio Sánchez	37e40dca85	Fix ambiguity in PPC for vec_splats call.	2022-11-14 18:58:16 +00:00
Charles Schlosser	82b152dbe7	Add signbit function	2022-11-04 00:31:20 +00:00
Antonio Sánchez	886aad1361	Disable patan for double on PPC.	2022-10-27 17:56:08 +00:00
Rasmus Munk Larsen	1414a76fa9	Only vectorize atan<double> for Altivec if VSX is available.	2022-10-03 22:06:58 +00:00
Rasmus Munk Larsen	c475228b28	Vectorize atan() for double.	2022-10-01 01:49:30 +00:00
Rasmus Munk Larsen	bd393e15c3	Vectorize acos, asin, and atan for float.	2022-08-29 19:49:33 +00:00
Charles Schlosser	e5af9f87f2	Vectorize pow for integer base / exponent types	2022-08-29 19:23:54 +00:00
Chip Kerchner	9e0afe0f02	Fix non-VSX PowerPC build	2022-08-08 18:18:17 +00:00
Chip Kerchner	84a9d6fac9	Fix use of Packet2d type for non-VSX.	2022-08-03 20:48:13 +00:00
Chip Kerchner	ce60a7be83	Partial Packet support for GEMM real-only (PowerPC). Also fix compilation warnings & errors for some conditions in new API.	2022-08-03 18:15:19 +00:00
Chip Kerchner	84cf3ff18d	Add pload_partial, pstore_partial (and unaligned versions), pgather_partial, pscatter_partial, loadPacketPartial and storePacketPartial.	2022-06-27 19:18:00 +00:00
Chip Kerchner	c603275dc9	Better performance for Power10 using more load and store vector pairs for GEMV	2022-06-27 18:11:55 +00:00
Chip Kerchner	4d1c16eab8	Fix tanh and erf to use vectorized version for EIGEN_FAST_MATH in VSX.	2022-06-15 16:06:43 +00:00
Chip Kerchner	aa8b7e2c37	Add subMappers to Power GEMM packing - simplifies the address calculations (10% faster)	2022-05-23 15:18:29 +00:00
Eisuke Kawashima	ac5c83a3f5	unset executable flag	2022-05-22 22:47:43 +09:00
Chip Kerchner	c2f15edc43	Add load vector_pairs for RHS of GEMM MMA. Improved predux GEMV.	2022-04-25 16:23:01 +00:00
Chip Kerchner	44ba7a0da3	Fix compiler bugs for GCC 10 & 11 for Power GEMM	2022-04-20 15:59:00 +00:00
Chip Kerchner	b02c384ef4	Add fused multiply functions for PowerPC - pmsub, pnmadd and pnmsub	2022-04-18 16:16:32 +00:00
Chip Kerchner	53eec53d2a	Fix Power GEMV order of operations in predux for MMA.	2022-04-11 21:29:05 +00:00
Chip Kerchner	403fa33409	Performance improvements in GEMM for Power	2022-04-05 12:18:53 +00:00
Chip Kerchner	0699fa06fe	Split general_matrix_vector_product interface for Power into two macros - one ColMajor and RowMajor.	2022-03-23 18:09:33 +00:00
Chip Kerchner	7b10795e39	Change EIGEN_ALTIVEC_ENABLE_MMA_DYNAMIC_DISPATCH and EIGEN_ALTIVEC_DISABLE_MMA flags to be like TensorFlow's...	2022-03-17 22:35:27 +00:00
Antonio Sanchez	e34db1239d	Fix missing pound	2022-03-16 12:26:12 -07:00
Antonio Sánchez	591906477b	Fix up PowerPC MMA flags so it builds by default.	2022-03-16 19:16:28 +00:00
Erik Schultheis	421cbf0866	Replace Eigen type metaprogramming with corresponding std types and make use of alias templates	2022-03-16 16:43:40 +00:00
Rasmus Munk Larsen	9ad5661482	Revert "Fix up PowerPC MMA flags so it builds by default."	2022-03-15 20:51:03 +00:00
Antonio Sánchez	65eeedf964	Fix up PowerPC MMA flags so it builds by default.	2022-03-15 20:22:23 +00:00
Sean McBride	f1b9692d63	Removed EIGEN_UNUSED decorations from many functions that are in fact used	2022-03-03 20:19:33 +00:00
Chip Kerchner	cb5ca1c901	Cleanup compiler warnings, etc from recent changes in GEMM & GEMV for PowerPC	2022-02-09 18:47:08 +00:00
Chip Kerchner	66464bd2a8	Fix number of block columns to NOT overflow the cache (PowerPC) abnormally in GEMV	2022-01-27 20:35:53 +00:00
Chip Kerchner	708fd6d136	Add MMA and performance improvements for VSX in GEMV for PowerPC.	2022-01-13 13:23:18 +00:00