Alexander Richardson
a5469a6f0f
Avoid including <sstream> with EIGEN_NO_IO
...
(cherry picked from commit b7668c0371 )
2023-07-07 15:21:17 -07:00
Antonio Sánchez
8a21df2d9c
Disable f16c scalar conversions for MSVC.
...
(cherry picked from commit 73b2c13bf2 )
2023-07-07 15:21:12 -07:00
Antonio Sánchez
973b04f3e1
Fix AVX512 builds with MSVC.
...
(cherry picked from commit 9a14d91a99 )
2023-07-07 15:06:18 -07:00
Antonio Sánchez
e0fe006915
Fix mixingtypes for g++-11.
...
(cherry picked from commit 19c39bea29 )
2023-07-07 11:47:23 -07:00
Antonio Sánchez
995714142d
Restrict GCC<6.3 maxpd workaround to only gcc.
...
(cherry picked from commit 4bffbe84f9 )
2023-07-07 11:39:27 -07:00
Antonio Sánchez
730a781221
Define EIGEN_HAS_AVX512_MATH in PacketMath.
...
(cherry picked from commit e7f4a901ee )
2023-07-07 11:39:13 -07:00
Antonio Sánchez
77b2807322
Fix AVX512 math function consistency, enable for ICC.
...
(cherry picked from commit 96da541cba )
2023-07-07 11:37:49 -07:00
Antonio Sánchez
52e545324e
Fix ODR violations.
...
(cherry picked from commit cafeadffef )
2023-07-07 11:37:31 -07:00
Chip Kerchner
fbdaff81bd
Invert rows and depth in non-vectorized portion of packing (PowerPC).
...
(cherry picked from commit 9cf34ee0ae )
2021-11-03 23:34:47 +00:00
Andreas Krebbel
23469c3cda
ZVector: Move alignas qualifier to come first
...
We currently have plenty of type definitions with the alignment
qualifier coming after the type. The compiler warns about ignoring
them:
int EIGEN_ALIGN16 ai[4];
Turn this into:
EIGEN_ALIGN16 int ai[4];
(cherry picked from commit 8faafc3aaa )
2021-11-03 23:29:10 +00:00
Antonio Sanchez
18824d10ea
Fix ZVector build.
...
Cross-compiled via `s390x-linux-gnu-g++`, run via qemu. This allows the
packetmath tests to pass.
(cherry picked from commit 40bbe8a4d0 )
2021-11-03 23:28:26 +00:00
Antonio Sanchez
943ef50a2d
Disable testing of complex compound assignment operators for MSVC.
...
MSVC does not support specializing compound assignments for
`std::complex`, since it already specializes them (contrary to the
standard).
Trying to use one of these on device will currently lead to a
duplicate definition error. This is still probably preferable
to no error though. If we remove the definitions for MSVC, then
it will compile, but the kernel will fail silently.
The only proper solution would be to define our own custom `Complex`
type.
(cherry picked from commit f0f1d7938b )
2021-10-11 10:00:29 -07:00
Alexander Grund
929bc0e191
Fix alias violation in BFloat16
...
reinterpret_cast between unrelated types is undefined behavior and leads
to misoptimizations on some platforms.
Use the safer (and faster) version via bit_cast
(cherry picked from commit b5eaa42695 )
2021-09-20 14:25:58 +00:00
Antonio Sanchez
f046e326d9
Fix strict aliasing bug causing product_small failure.
...
Packet loading is skipped due to aliasing violation, leading to nullopt matrix
multiplication.
Fixes #2327 .
(cherry picked from commit 3c724c44cf )
2021-09-19 18:06:17 +00:00
Antonio Sanchez
4ef67cbfb2
GCC 4.8 arm EIGEN_OPTIMIZATION_BARRIER fix ( #2315 ).
...
GCC 4.8 doesn't seem to like the `g` register constraint, failing to
compile with "error: 'asm' operand requires impossible reload".
Tested `r` instead, and that seems to work, even with latest compilers.
Also fixed some minor macro issues to eliminate warnings on armv7.
Fixes #2315 .
(cherry picked from commit ff07a8a639 )
2021-08-31 21:23:28 +00:00
Antonio Sanchez
c2b6df6e60
Disable cuda Eigen::half vectorization on host.
...
All cuda `__half` functions are device-only in CUDA 9, including
conversions. Host-side conversions were added in CUDA 10.
The existing code doesn't build prior to 10.0.
All arithmetic functions are always device-only, so there's
therefore no reason to use vectorization on the host at all.
Modified the code to disable vectorization for `__half` on host,
which required also updating the `TensorReductionGpu` implementation
which previously made assumptions about available packets.
(cherry picked from commit cc3573ab44 )
2021-08-31 21:23:11 +00:00
Antonio Sanchez
f1032255d3
Add missing PPC packet comparisons.
...
This is to fix the packetmath tests on the ppc pipeline.
(cherry picked from commit 2cc6ee0d2e )
2021-08-17 15:33:55 +00:00
Chip-Kerchner
f57dec64ef
Fix unaligned loads in ploadLhs & ploadRhs for P8.
...
(cherry picked from commit 8dcf3e38ba )
2021-08-17 12:48:36 +00:00
Chip-Kerchner
0b56b62f30
Reverse compare logic in F32ToBf16 since vec_cmpne is not available in Power8 - now compiles for clang10 default (P8).
...
(cherry picked from commit e07227c411 )
2021-08-13 18:01:15 +00:00
Chip Kerchner
44cc96e1a1
Get rid of used uninitialized warnings for EIGEN_UNUSED_VARIABLE in gcc11+
...
(cherry picked from commit 66499f0f17 )
2021-08-12 21:39:17 +00:00
ChipKerchner
13d7658c5d
Fix errors on older compilers (gcc 7.5 - lack of vec_neg, clang10 - can not use const pointers with vec_xl).
...
(cherry picked from commit 413bc491f1 )
2021-08-10 20:40:54 +00:00
Gauri Deshpande
93bff85a42
remove denormal flushing in fp32tobf16 for avx & avx512
...
(cherry picked from commit e6a5a594a7 )
2021-08-09 22:15:42 +00:00
Rasmus Munk Larsen
05bab8139a
Fix breakage of conj_helper in conjunction with custom types introduced in !537 .
...
(cherry picked from commit 7b35638ddb )
2021-07-02 20:59:50 +00:00
Chip Kerchner
eebde572d9
Create the ability to disable the specialized gemm_pack_rhs in Eigen (only PPC) for TensorFlow
...
(cherry picked from commit 91e99ec1e0 )
2021-07-01 23:32:38 +00:00
大河メタル
94e2250b36
Correct declarations for aarch64-pc-windows-msvc
...
(cherry picked from commit c81da59a25 )
2021-06-30 04:10:04 +00:00
Rasmus Munk Larsen
380d0e4916
Get rid of redundant pabs instruction in complex square root.
...
(cherry picked from commit 5aebbe9098 )
2021-06-29 23:27:09 +00:00
Rohit Santhanam
e83af2cc24
Commit 52a5f982 broke conjhelper functionality for HIP GPUs.
...
This commit addresses this.
(cherry picked from commit 2d132d1736 )
2021-06-25 19:56:18 +00:00
Rasmus Munk Larsen
413ff2b531
Small cleanup: Get rid of the macros EIGEN_HAS_SINGLE_INSTRUCTION_CJMADD and CJMADD, which were effectively unused, apart from on x86, where the change results in identically performing code.
...
(cherry picked from commit bffd267d17 )
2021-06-25 17:13:12 +00:00
Rasmus Munk Larsen
a235ddef39
Get rid of code duplication for conj_helper. For packets where LhsType=RhsType a single generic implementation suffices. For scalars, the generic implementation of pconj automatically forwards to numext::conj, so much of the existing specialization can be avoided. For mixed types we still need specializations.
...
(cherry picked from commit 52a5f98212 )
2021-06-24 23:30:42 +00:00
Antonio Sanchez
ee4e099aa2
Remove pset, replace with ploadu.
...
We can't make guarantees on alignment for existing calls to `pset`,
so we should default to loading unaligned. But in that case, we should
just use `ploadu` directly. For loading constants, this load should hopefully
get optimized away.
This is causing segfaults in Google Maps.
(cherry picked from commit 12e8d57108 )
2021-06-17 17:11:08 +00:00
Chip-Kerchner
9fc93ce31a
EIGEN_STRONG_INLINE was NOT inlining in some critical needed areas (6.6X slowdown) when used with Tensorflow. Changing to EIGEN_ALWAYS_INLINE where appropiate.
...
(cherry picked from commit ef1fd341a8 )
2021-06-16 22:14:17 +00:00
Antonio Sanchez
1374f49f28
Add missing ppc pcmp_lt_or_nan<Packet8bf>
...
(cherry picked from commit 9e94c59570 )
2021-06-15 22:12:22 +00:00
Rasmus Munk Larsen
1cb1ffd5b2
Use bit_cast to create -0.0 for floating point types to avoid compiler optimization changing sign with --ffast-math enabled.
...
(cherry picked from commit fc87e2cbaa )
2021-06-11 02:57:02 +00:00
Antonio Sanchez
98cf1e076f
Add missing NEON ptranspose implementations.
...
Unified implementation using only `vzip`.
(cherry picked from commit dba753a986 )
2021-05-25 19:09:50 +00:00
guoqiangqi
2f908f8255
Changing the storage of the SSE complex packets to that of the wrapper. This should fix #2242 .
...
(cherry picked from commit 3d9051ea84 )
2021-05-12 17:02:19 +00:00
Christoph Hertzberg
9e0dc8f09b
Revert addition of unused paddsub<Packet2cf>. This fixes #2242
...
(cherry picked from commit 722ca0b665 )
2021-05-07 16:23:03 +00:00
Antonio Sanchez
fc2cc10842
Better CUDA complex division.
...
The original produced NaNs when dividing 0/b for subnormal b.
The `complex_divide_stable` was changed to use the more common
Smith's algorithm.
(cherry picked from commit 1c013be2cc )
2021-04-29 17:58:45 +00:00
Antonio Sanchez
a33855f6ee
Add missing pcmp_lt_or_nan for NEON Packet4bf.
...
(cherry picked from commit 172db7bfc3 )
2021-04-27 21:15:08 +00:00
Jakub Lichman
ac3c5aad31
Tests added and AVX512 bug fixed for pcmp_lt_or_nan
...
(cherry picked from commit d87648a6be )
2021-04-26 18:07:55 +00:00
Jakub Lichman
42a8bdd4d7
HasExp added for AVX512 Packet8d
...
(cherry picked from commit 2b1dfd1ba0 )
2021-04-21 12:09:21 +02:00
Chip-Kerchner
28564957ac
Fix taking address of rvalue compiler issue with TensorFlow (plus other warnings).
...
(cherry picked from commit 06c2760bd1 )
2021-04-21 01:05:21 +00:00
Antonio Sanchez
ab7fe215f9
Fix ldexp for AVX512 ( #2215 )
...
Wrong shuffle was used. Need to interleave low/high halves with a
`permute` instruction.
Fixes #2215 .
(cherry picked from commit 1d79c68ba0 )
2021-04-20 20:52:26 +00:00
Christoph Hertzberg
9357feedc7
Avoid using uninitialized inputs and if available, use slightly more efficient movsd instruction for pset1<Packet2cf>.
2021-04-13 01:36:59 +02:00
Chip Kerchner
c24bee6120
Fix address of temporary object errors in clang11.
...
This fixes the problem with taking the address of temporary objects which clang11 treats as errors.
2021-04-02 16:27:08 +00:00
Antonio Sanchez
87729ea39f
Eliminate round_impl double-promotion warnings for c++03.
2021-03-25 16:52:19 +00:00
Chip Kerchner
d59ef212e1
Fixed performance issues for complex VSX and P10 MMA in gebp_kernel (level 3).
2021-03-25 11:08:19 +00:00
Christoph Hertzberg
69a4f70956
Revert "Uses _mm512_abs_pd for Packet8d pabs"
...
This reverts commit f019b97aca
2021-03-23 18:52:19 +00:00
David Tellenbach
4811e81966
Remove yet another comma at end of enum
2021-03-18 23:30:00 +01:00
Steve Bronder
f019b97aca
Uses _mm512_abs_pd for Packet8d pabs
2021-03-18 15:47:52 +00:00
Antonio Sanchez
8dfe1029a5
Augment NumTraits with min/max_exponent() again.
...
Replace usage of `std::numeric_limits<...>::min/max_exponent` in
codebase where possible. Also replaced some other `numeric_limits`
usages in affected tests with the `NumTraits` equivalent.
The previous MR !443 failed for c++03 due to lack of `constexpr`.
Because of this, we need to keep around the `std::numeric_limits`
version in enum expressions until the switch to c++11.
Fixes #2148
2021-03-16 20:12:46 -07:00