Rasmus Munk Larsen
3be5ee2352
Update copy helper to use fast_memcpy.
2017-01-24 14:22:49 -08:00
Rasmus Munk Larsen
e6b1020221
Adds a fast memcpy function to Eigen. This takes advantage of the following:
...
1. For small fixed sizes, the compiler generates inline code for memcpy, which is much faster.
2. My colleague eriche at googl dot com discovered that for large sizes, memmove is significantly faster than memcpy (at least on Linux with GCC or Clang). See benchmark numbers measured on a Haswell (HP Z440) workstation here: https://docs.google.com/a/google.com/spreadsheets/d/1jLs5bKzXwhpTySw65MhG1pZpsIwkszZqQTjwrd_n0ic/pubhtml This is of course surprising since memcpy is a less constrained version of memmove. This stackoverflow thread contains some speculation as to the causes: http://stackoverflow.com/questions/22793669/poor-memcpy-performance-on-linux
Below are numbers for copying and slicing tensors using the multithreaded TensorDevice. The numbers show significant improvements for memcpy of very small blocks and for memcpy of large blocks single threaded (we were already able to saturate memory bandwidth for >1 threads before on large blocks). The "slicingSmallPieces" benchmark also shows small consistent improvements, since memcpy cost is a fair portion of that particular computation.
The benchmarks operate on NxN matrices, and the names are of the form BM_$OP_${NUMTHREADS}T/${N}.
Measured improvements in wall clock time:
Run on rmlarsen3.mtv (12 X 3501 MHz CPUs); 2017-01-20T11:26:31.493023454-08:00
CPU: Intel Haswell with HyperThreading (6 cores) dL1:32KB dL2:256KB dL3:15MB
Benchmark Base (ns) New (ns) Improvement
------------------------------------------------------------------
BM_memcpy_1T/2 3.48 2.39 +31.3%
BM_memcpy_1T/8 12.3 6.51 +47.0%
BM_memcpy_1T/64 371 383 -3.2%
BM_memcpy_1T/512 66922 66720 +0.3%
BM_memcpy_1T/4k 9892867 6849682 +30.8%
BM_memcpy_1T/5k 14951099 10332856 +30.9%
BM_memcpy_2T/2 3.50 2.46 +29.7%
BM_memcpy_2T/8 12.3 7.66 +37.7%
BM_memcpy_2T/64 371 376 -1.3%
BM_memcpy_2T/512 66652 66788 -0.2%
BM_memcpy_2T/4k 6145012 6117776 +0.4%
BM_memcpy_2T/5k 9181478 9010942 +1.9%
BM_memcpy_4T/2 3.47 2.47 +31.0%
BM_memcpy_4T/8 12.3 6.67 +45.8
BM_memcpy_4T/64 374 376 -0.5%
BM_memcpy_4T/512 67833 68019 -0.3%
BM_memcpy_4T/4k 5057425 5188253 -2.6%
BM_memcpy_4T/5k 7555638 7779468 -3.0%
BM_memcpy_6T/2 3.51 2.50 +28.8%
BM_memcpy_6T/8 12.3 7.61 +38.1%
BM_memcpy_6T/64 373 378 -1.3%
BM_memcpy_6T/512 66871 66774 +0.1%
BM_memcpy_6T/4k 5112975 5233502 -2.4%
BM_memcpy_6T/5k 7614180 7772246 -2.1%
BM_memcpy_8T/2 3.47 2.41 +30.5%
BM_memcpy_8T/8 12.4 10.5 +15.3%
BM_memcpy_8T/64 372 388 -4.3%
BM_memcpy_8T/512 67373 66588 +1.2%
BM_memcpy_8T/4k 5148462 5254897 -2.1%
BM_memcpy_8T/5k 7660989 7799058 -1.8%
BM_memcpy_12T/2 3.50 2.40 +31.4%
BM_memcpy_12T/8 12.4 7.55 +39.1
BM_memcpy_12T/64 374 378 -1.1%
BM_memcpy_12T/512 67132 66683 +0.7%
BM_memcpy_12T/4k 5185125 5292920 -2.1%
BM_memcpy_12T/5k 7717284 7942684 -2.9%
BM_slicingSmallPieces_1T/2 47.3 47.5 +0.4%
BM_slicingSmallPieces_1T/8 53.6 52.3 +2.4%
BM_slicingSmallPieces_1T/64 491 476 +3.1%
BM_slicingSmallPieces_1T/512 21734 18814 +13.4%
BM_slicingSmallPieces_1T/4k 394660 396760 -0.5%
BM_slicingSmallPieces_1T/5k 218722 209244 +4.3%
BM_slicingSmallPieces_2T/2 80.7 79.9 +1.0%
BM_slicingSmallPieces_2T/8 54.2 53.1 +2.0
BM_slicingSmallPieces_2T/64 497 477 +4.0%
BM_slicingSmallPieces_2T/512 21732 18822 +13.4%
BM_slicingSmallPieces_2T/4k 392885 390490 +0.6%
BM_slicingSmallPieces_2T/5k 221988 208678 +6.0%
BM_slicingSmallPieces_4T/2 80.8 80.1 +0.9%
BM_slicingSmallPieces_4T/8 54.1 53.2 +1.7%
BM_slicingSmallPieces_4T/64 493 476 +3.4%
BM_slicingSmallPieces_4T/512 21702 18758 +13.6%
BM_slicingSmallPieces_4T/4k 393962 404023 -2.6%
BM_slicingSmallPieces_4T/5k 249667 211732 +15.2%
BM_slicingSmallPieces_6T/2 80.5 80.1 +0.5%
BM_slicingSmallPieces_6T/8 54.4 53.4 +1.8%
BM_slicingSmallPieces_6T/64 488 478 +2.0%
BM_slicingSmallPieces_6T/512 21719 18841 +13.3%
BM_slicingSmallPieces_6T/4k 394950 397583 -0.7%
BM_slicingSmallPieces_6T/5k 223080 210148 +5.8%
BM_slicingSmallPieces_8T/2 81.2 80.4 +1.0%
BM_slicingSmallPieces_8T/8 58.1 53.5 +7.9%
BM_slicingSmallPieces_8T/64 489 480 +1.8%
BM_slicingSmallPieces_8T/512 21586 18798 +12.9%
BM_slicingSmallPieces_8T/4k 394592 400165 -1.4%
BM_slicingSmallPieces_8T/5k 219688 208301 +5.2%
BM_slicingSmallPieces_12T/2 80.2 79.8 +0.7%
BM_slicingSmallPieces_12T/8 54.4 53.4 +1.8
BM_slicingSmallPieces_12T/64 488 476 +2.5%
BM_slicingSmallPieces_12T/512 21931 18831 +14.1%
BM_slicingSmallPieces_12T/4k 393962 396541 -0.7%
BM_slicingSmallPieces_12T/5k 218803 207965 +5.0%
2017-01-24 13:55:18 -08:00
Gael Guennebaud
ca79c1545a
Add std:: namespace prefix to all (hopefully) instances if size_t/ptrdfiff_t
2017-01-23 22:02:53 +01:00
Gael Guennebaud
e383d6159a
MSVC 2015 has all we want about c++11 and MSVC 2017 fails on binder1st/binder2nd
2017-01-06 15:44:13 +01:00
Gael Guennebaud
d32a43e33a
Make sure that traits<CwiseBinaryOp>::Flags reports the correct storage order so that methods like .outerSize()/.innerSize() work properly.
2016-12-27 16:35:45 +01:00
Gael Guennebaud
6b8f637ab1
Harmless typo
2016-12-27 16:31:17 +01:00
Gael Guennebaud
f2f9df8aa5
Remove MSVC warning 4127 - conditional expression is constant from the disabled list as we now have a local workaround.
2016-12-20 22:53:19 +01:00
Gael Guennebaud
2b3fc981b8
bug #1362 : workaround constant conditional warning produced by MSVC
2016-12-20 22:52:27 +01:00
Gael Guennebaud
94e8d8902f
Fix bug #1367 : compilation fix for gcc 4.1!
2016-12-20 22:17:01 +01:00
Benoit Steiner
9e03dfb452
Made sure EIGEN_HAS_C99_MATH is defined when compiling OpenCL code
2016-12-17 09:23:37 -08:00
Benoit Steiner
1324ffef2f
Reenabled the use of constexpr on OpenCL devices
2016-12-15 06:49:38 -08:00
Gael Guennebaud
3abc827354
Clean debugging code
2016-12-05 12:59:32 +01:00
Gael Guennebaud
6a5fe86098
Complete rewrite of column-major-matrix * vector product to deliver higher performance of modern CPU.
...
The previous code has been optimized for Intel core2 for which unaligned loads/stores were prohibitively expensive.
This new version exhibits much higher instruction independence (better pipelining) and explicitly leverage FMA.
According to my benchmark, on Haswell this new kernel is always faster than the previous one, and sometimes even twice as fast.
Even higher performance could be achieved with a better blocking size heuristic and, perhaps, with explicit prefetching.
We should also check triangular product/solve to optimally exploit this new kernel (working on vertical panel of 4 columns is probably not optimal anymore).
2016-12-03 21:14:14 +01:00
Angelos Mantzaflaris
8c24723a09
typo UIntPtr
...
(grafted from b6f04a2dd4
2016-12-01 21:25:58 +01:00
Angelos Mantzaflaris
aeba0d8655
fix two warnings(unused typedef, unused variable) and a typo
...
(grafted from a9aa3bcf50
2016-12-01 21:23:43 +01:00
Benoit Steiner
fd1dc3363e
Merged eigen/eigen into default
2016-11-30 20:16:17 -08:00
Gael Guennebaud
c927af60ed
Fix a performance regression in (mat*mat)*vec for which mat*mat was evaluated multiple times.
2016-11-30 17:59:13 +01:00
Benoit Steiner
9f8fbd9434
Merged eigen/eigen into default
2016-11-26 11:28:25 -08:00
Mehdi Goli
7318daf887
Fixing LLVM error on TensorMorphingSycl.h on GPU; fixing int64_t crash for tensor_broadcast_sycl on GPU; adding get_sycl_supported_devices() on syclDevice.h.
2016-11-25 16:19:07 +00:00
Benoit Steiner
3be1afca11
Disabled the "remove the call to 'std::abs' since unsigned values cannot be negative" warning introduced in clang 3.5
2016-11-23 18:49:51 -08:00
Mehdi Goli
b8cc5635d5
Removing unsupported device from test case; cleaning the tensor device sycl.
2016-11-23 16:30:41 +00:00
Benoit Steiner
ed839c5851
Enable the use of constant expressions with clang >= 3.6
2016-11-20 10:34:49 -08:00
Benoit Steiner
8649e16c2a
Enable EIGEN_HAS_C99_MATH when building with the latest version of Visual Studio
2016-11-18 14:18:34 -08:00
Chun Wang
0d0948c3b9
Workaround for error in VS2012 with /clr
2016-11-17 17:54:27 -05:00
Gael Guennebaud
7b09e4dd8c
bump default branch to 3.3.90
2016-11-16 22:20:58 +01:00
Gael Guennebaud
eeac81b8c0
bump to 3.3.0
2016-11-10 13:55:14 +01:00
Gael Guennebaud
ba05572dcb
bump to 3.3-rc2
2016-11-04 09:09:06 +01:00
Benoit Steiner
ca0ba0d9a4
Improved AVX512 support
2016-11-03 04:00:49 -07:00
Benoit Steiner
c80587c92b
Merged eigen/eigen into default
2016-11-03 03:55:11 -07:00
Gael Guennebaud
78e93ac1ad
bug #1330 : Cholmod supports double precision only, so let's trigger a static assertion if the scalar type does not match this requirement.
2016-11-03 10:21:59 +01:00
Gael Guennebaud
97feea9d39
add a generic EIGEN_HAS_CXX11
2016-10-26 15:53:13 +02:00
Gael Guennebaud
40f62974b7
bug #1328 : workaround a compilation issue with gcc 4.2
2016-10-20 19:19:37 +02:00
Avi Ginsburg
ac63d6891c
Patch to allow VS2015 & CUDA 8.0 to compile with Eigen included. I'm not sure
...
whether to limit the check to this compiler combination
(` || (EIGEN_COMP_MSVC == 1900 && __CUDACC_VER__) `)
or to leave it as it is. I also don't know if this will have any affect on
including Eigen in device code (I'm not in my current project).
2016-10-13 08:47:32 +00:00
Benoit Steiner
78d2926508
Merged eigen/eigen into default
2016-10-12 13:46:29 -07:00
Benoit Steiner
5727e4d89c
Reenabled the use of variadic templates on tegra x1 provides that the latest version (i.e. JetPack 2.3) is used.
2016-10-08 22:19:03 +00:00
Benoit Steiner
78b569f685
Merged latest updates from trunk
2016-10-05 18:48:55 -07:00
Benoit Steiner
27d7628f16
Updated the list of warnings to reflect the new message ids introduced in cuda 8.0
2016-09-28 17:42:59 -07:00
Sergiu Deitsch
fe29157d02
disabled MSVC level 4 warning C4714
...
The level 4 warning (/W4) warns about functions marked as __forceinline not
inlined, and generates a lot of noise.
2016-09-25 14:25:47 +02:00
Gael Guennebaud
77e27fbeee
bump to 3.3-rc1
2016-09-22 22:37:39 +02:00
Gael Guennebaud
ac5377e161
Improve cost estimation of complex division
2016-09-21 17:26:04 +02:00
Benoit Steiner
779faaaeba
Fixed compilation warnings generated by nvcc 6.5 (and below) when compiling the EIGEN_THROW macro
2016-09-14 09:56:11 -07:00
Benoit Steiner
ff47717f25
Suppress warning 2527 and 2529, which correspond to the "calling a __host__ function from a __host__ __device__ function is not allowed" message in nvcc 6.5.
2016-09-13 12:49:40 -07:00
Benoit Steiner
309190cf02
Suppress message 1222 when compiling with nvcc: this ensures that we don't warnings about unknown warning messages when compiling with older versions of nvcc
2016-09-13 12:42:13 -07:00
Gael Guennebaud
228ae29591
Fix compilation on 32 bits systems.
2016-09-09 22:34:38 +02:00
Gael Guennebaud
471eac5399
bug #1195 : move NumTraits::Div<>::Cost to internal::scalar_div_cost (with some specializations in arch/SSE and arch/AVX)
2016-09-08 08:36:27 +02:00
Gael Guennebaud
85fb517eaf
Generalize ScalarBinaryOpTraits to any complex-real combination as defined by NumTraits (instead of supporting std::complex only).
2016-09-06 17:23:15 +02:00
Gael Guennebaud
e13071dd13
Workaround a weird msvc 2012 compilation error.
2016-09-05 15:50:41 +02:00
Benoit Steiner
373c340b71
Fixed a typo
2016-09-02 15:41:17 -07:00
Gael Guennebaud
d6c8366d84
Fix compilation with MSVC 2012
2016-09-02 15:23:32 +02:00
Gael Guennebaud
ef54723dbe
One more msvc fix iteration, the previous one was over-simplified for visual
2016-09-01 15:04:53 +02:00
