Commit Graph

1077 Commits

Author SHA1 Message Date
Antonio Sánchez
99473f255b Fix failing MSVC tests due to compiler bugs.
(cherry picked from commit 394aabb0a3)
2023-07-10 14:52:08 -07:00
Antonio Sánchez
dae8c6d7ad Guard complex sqrt on old MSVC compilers.
(cherry picked from commit a16fb889dd)
2023-07-10 14:52:07 -07:00
Antonio Sánchez
2dfdaa2abf More NEON packetmath fixes.
(cherry picked from commit 384269937f)
2023-07-10 14:52:03 -07:00
Antonio Sánchez
a659b5dbb2 Fix NEON make_packet2f.
(cherry picked from commit 2dfbf1b251)
2023-07-10 14:34:09 -07:00
Antonio Sánchez
879854382c Fix MSVC arm build.
(cherry picked from commit 0a5392d606)
2023-07-10 14:34:09 -07:00
Martin Burchell
b26ada1e03 Fix error: unused parameter 'tmp' [-Werror,-Wunused-parameter] on clang/32-bit arm
(cherry picked from commit c54785b071)
2023-07-10 14:34:09 -07:00
Antonio Sánchez
5547205092 Correct pnegate for floating-point zero.
(cherry picked from commit 8588d8c74b)
2023-07-10 14:34:04 -07:00
Matthew Sterrett
d0e2b3e58d Removed unnecessary checks for FP16C
(cherry picked from commit 39fcc89798)
2023-07-07 15:21:17 -07:00
Lexi Bromfield
33a602eb37 Don't double-define Half functions on aarch64
(cherry picked from commit 66ea0c09fd)
2023-07-07 15:21:17 -07:00
Rasmus Munk Larsen
a9490cd3c5 Fix code and unit test for a few corner cases in vectorized pow()
(cherry picked from commit 7a87ed1b6a)
2023-07-07 15:21:17 -07:00
Alexander Richardson
a5469a6f0f Avoid including <sstream> with EIGEN_NO_IO
(cherry picked from commit b7668c0371)
2023-07-07 15:21:17 -07:00
Antonio Sánchez
8a21df2d9c Disable f16c scalar conversions for MSVC.
(cherry picked from commit 73b2c13bf2)
2023-07-07 15:21:12 -07:00
Antonio Sánchez
973b04f3e1 Fix AVX512 builds with MSVC.
(cherry picked from commit 9a14d91a99)
2023-07-07 15:06:18 -07:00
Antonio Sánchez
e0fe006915 Fix mixingtypes for g++-11.
(cherry picked from commit 19c39bea29)
2023-07-07 11:47:23 -07:00
Antonio Sánchez
995714142d Restrict GCC<6.3 maxpd workaround to only gcc.
(cherry picked from commit 4bffbe84f9)
2023-07-07 11:39:27 -07:00
Antonio Sánchez
730a781221 Define EIGEN_HAS_AVX512_MATH in PacketMath.
(cherry picked from commit e7f4a901ee)
2023-07-07 11:39:13 -07:00
Antonio Sánchez
77b2807322 Fix AVX512 math function consistency, enable for ICC.
(cherry picked from commit 96da541cba)
2023-07-07 11:37:49 -07:00
Antonio Sánchez
52e545324e Fix ODR violations.
(cherry picked from commit cafeadffef)
2023-07-07 11:37:31 -07:00
Chip Kerchner
fbdaff81bd Invert rows and depth in non-vectorized portion of packing (PowerPC).
(cherry picked from commit 9cf34ee0ae)
2021-11-03 23:34:47 +00:00
Andreas Krebbel
23469c3cda ZVector: Move alignas qualifier to come first
We currently have plenty of type definitions with the alignment
qualifier coming after the type.  The compiler warns about ignoring
them:
int EIGEN_ALIGN16 ai[4];

Turn this into:
EIGEN_ALIGN16 int ai[4];


(cherry picked from commit 8faafc3aaa)
2021-11-03 23:29:10 +00:00
Antonio Sanchez
18824d10ea Fix ZVector build.
Cross-compiled via `s390x-linux-gnu-g++`, run via qemu.  This allows the
packetmath tests to pass.


(cherry picked from commit 40bbe8a4d0)
2021-11-03 23:28:26 +00:00
Antonio Sanchez
943ef50a2d Disable testing of complex compound assignment operators for MSVC.
MSVC does not support specializing compound assignments for
`std::complex`, since it already specializes them (contrary to the
standard).

Trying to use one of these on device will currently lead to a
duplicate definition error.  This is still probably preferable
to no error though.  If we remove the definitions for MSVC, then
it will compile, but the kernel will fail silently.

The only proper solution would be to define our own custom `Complex`
type.

(cherry picked from commit f0f1d7938b)
2021-10-11 10:00:29 -07:00
Alexander Grund
929bc0e191 Fix alias violation in BFloat16
reinterpret_cast between unrelated types is undefined behavior and leads
to misoptimizations on some platforms.
Use the safer (and faster) version via bit_cast


(cherry picked from commit b5eaa42695)
2021-09-20 14:25:58 +00:00
Antonio Sanchez
f046e326d9 Fix strict aliasing bug causing product_small failure.
Packet loading is skipped due to aliasing violation, leading to nullopt matrix
multiplication.

Fixes #2327.


(cherry picked from commit 3c724c44cf)
2021-09-19 18:06:17 +00:00
Antonio Sanchez
4ef67cbfb2 GCC 4.8 arm EIGEN_OPTIMIZATION_BARRIER fix (#2315).
GCC 4.8 doesn't seem to like the `g` register constraint, failing to
compile with "error: 'asm' operand requires impossible reload".

Tested `r` instead, and that seems to work, even with latest compilers.

Also fixed some minor macro issues to eliminate warnings on armv7.

Fixes #2315.


(cherry picked from commit ff07a8a639)
2021-08-31 21:23:28 +00:00
Antonio Sanchez
c2b6df6e60 Disable cuda Eigen::half vectorization on host.
All cuda `__half` functions are device-only in CUDA 9, including
conversions. Host-side conversions were added in CUDA 10.
The existing code doesn't build prior to 10.0.

All arithmetic functions are always device-only, so there's
therefore no reason to use vectorization on the host at all.

Modified the code to disable vectorization for `__half` on host,
which required also updating the `TensorReductionGpu` implementation
which previously made assumptions about available packets.


(cherry picked from commit cc3573ab44)
2021-08-31 21:23:11 +00:00
Antonio Sanchez
f1032255d3 Add missing PPC packet comparisons.
This is to fix the packetmath tests on the ppc pipeline.


(cherry picked from commit 2cc6ee0d2e)
2021-08-17 15:33:55 +00:00
Chip-Kerchner
f57dec64ef Fix unaligned loads in ploadLhs & ploadRhs for P8.
(cherry picked from commit 8dcf3e38ba)
2021-08-17 12:48:36 +00:00
Chip-Kerchner
0b56b62f30 Reverse compare logic ƒin F32ToBf16 since vec_cmpne is not available in Power8 - now compiles for clang10 default (P8).
(cherry picked from commit e07227c411)
2021-08-13 18:01:15 +00:00
Chip Kerchner
44cc96e1a1 Get rid of used uninitialized warnings for EIGEN_UNUSED_VARIABLE in gcc11+
(cherry picked from commit 66499f0f17)
2021-08-12 21:39:17 +00:00
ChipKerchner
13d7658c5d Fix errors on older compilers (gcc 7.5 - lack of vec_neg, clang10 - can not use const pointers with vec_xl).
(cherry picked from commit 413bc491f1)
2021-08-10 20:40:54 +00:00
Gauri Deshpande
93bff85a42 remove denormal flushing in fp32tobf16 for avx & avx512
(cherry picked from commit e6a5a594a7)
2021-08-09 22:15:42 +00:00
Rasmus Munk Larsen
05bab8139a Fix breakage of conj_helper in conjunction with custom types introduced in !537.
(cherry picked from commit 7b35638ddb)
2021-07-02 20:59:50 +00:00
Chip Kerchner
eebde572d9 Create the ability to disable the specialized gemm_pack_rhs in Eigen (only PPC) for TensorFlow
(cherry picked from commit 91e99ec1e0)
2021-07-01 23:32:38 +00:00
大河メタル
94e2250b36 Correct declarations for aarch64-pc-windows-msvc
(cherry picked from commit c81da59a25)
2021-06-30 04:10:04 +00:00
Rasmus Munk Larsen
380d0e4916 Get rid of redundant pabs instruction in complex square root.
(cherry picked from commit 5aebbe9098)
2021-06-29 23:27:09 +00:00
Rohit Santhanam
e83af2cc24 Commit 52a5f982 broke conjhelper functionality for HIP GPUs.
This commit addresses this.


(cherry picked from commit 2d132d1736)
2021-06-25 19:56:18 +00:00
Rasmus Munk Larsen
413ff2b531 Small cleanup: Get rid of the macros EIGEN_HAS_SINGLE_INSTRUCTION_CJMADD and CJMADD, which were effectively unused, apart from on x86, where the change results in identically performing code.
(cherry picked from commit bffd267d17)
2021-06-25 17:13:12 +00:00
Rasmus Munk Larsen
a235ddef39 Get rid of code duplication for conj_helper. For packets where LhsType=RhsType a single generic implementation suffices. For scalars, the generic implementation of pconj automatically forwards to numext::conj, so much of the existing specialization can be avoided. For mixed types we still need specializations.
(cherry picked from commit 52a5f98212)
2021-06-24 23:30:42 +00:00
Antonio Sanchez
ee4e099aa2 Remove pset, replace with ploadu.
We can't make guarantees on alignment for existing calls to `pset`,
so we should default to loading unaligned.  But in that case, we should
just use `ploadu` directly. For loading constants, this load should hopefully
get optimized away.

This is causing segfaults in Google Maps.


(cherry picked from commit 12e8d57108)
2021-06-17 17:11:08 +00:00
Chip-Kerchner
9fc93ce31a EIGEN_STRONG_INLINE was NOT inlining in some critical needed areas (6.6X slowdown) when used with Tensorflow. Changing to EIGEN_ALWAYS_INLINE where appropiate.
(cherry picked from commit ef1fd341a8)
2021-06-16 22:14:17 +00:00
Antonio Sanchez
1374f49f28 Add missing ppc pcmp_lt_or_nan<Packet8bf>
(cherry picked from commit 9e94c59570)
2021-06-15 22:12:22 +00:00
Rasmus Munk Larsen
1cb1ffd5b2 Use bit_cast to create -0.0 for floating point types to avoid compiler optimization changing sign with --ffast-math enabled.
(cherry picked from commit fc87e2cbaa)
2021-06-11 02:57:02 +00:00
Antonio Sanchez
98cf1e076f Add missing NEON ptranspose implementations.
Unified implementation using only `vzip`.


(cherry picked from commit dba753a986)
2021-05-25 19:09:50 +00:00
guoqiangqi
2f908f8255 Changing the storage of the SSE complex packets to that of the wrapper. This should fix #2242 .
(cherry picked from commit 3d9051ea84)
2021-05-12 17:02:19 +00:00
Christoph Hertzberg
9e0dc8f09b Revert addition of unused paddsub<Packet2cf>. This fixes #2242
(cherry picked from commit 722ca0b665)
2021-05-07 16:23:03 +00:00
Antonio Sanchez
fc2cc10842 Better CUDA complex division.
The original produced NaNs when dividing 0/b for subnormal b.
The `complex_divide_stable` was changed to use the more common
Smith's algorithm.


(cherry picked from commit 1c013be2cc)
2021-04-29 17:58:45 +00:00
Antonio Sanchez
a33855f6ee Add missing pcmp_lt_or_nan for NEON Packet4bf.
(cherry picked from commit 172db7bfc3)
2021-04-27 21:15:08 +00:00
Jakub Lichman
ac3c5aad31 Tests added and AVX512 bug fixed for pcmp_lt_or_nan
(cherry picked from commit d87648a6be)
2021-04-26 18:07:55 +00:00
Jakub Lichman
42a8bdd4d7 HasExp added for AVX512 Packet8d
(cherry picked from commit 2b1dfd1ba0)
2021-04-21 12:09:21 +02:00