devtools/eigen
1
0
Fork 0
mirror of https://gitlab.com/libeigen/eigen.git synced 2026-04-10 11:34:33 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,588 Commits 15 Branches 93 Tags
6d432eae5df88fe249d4db10c31647fa27872c8e
Commit Graph

185 Commits

Author SHA1 Message Date
Rasmus Munk Larsen
6d432eae5d Make is_valid_index_type return false for float and double when EIGEN_HAS_TYPE_TRAITS is off. 2019-06-05 16:42:27 -07:00
Eugene Zhulenev
e9f0eb8a5e Add masked_store_available to unpacket_traits 2019-05-02 14:52:58 -07:00
Anuj Rawat
8c7a6feb8e Adding lowlevel APIs for optimized RHS packet load in TensorFlow 
SpatialConvolution

Low-level APIs are added in order to optimized packet load in gemm_pack_rhs
in TensorFlow SpatialConvolution. The optimization is for scenario when a
packet is split across 2 adjacent columns. In this case we read it as two
'partial' packets and then merge these into 1. Currently this only works for
Packet16f (AVX512) and Packet8f (AVX2). We plan to add this for other
packet types (such as Packet8d) also.

This optimization shows significant speedup in SpatialConvolution with
certain parameters. Some examples are below.

Benchmark parameters are specified as:
Batch size, Input dim, Depth, Num of filters, Filter dim

Speedup numbers are specified for number of threads 1, 2, 4, 8, 16.

AVX512:

Parameters                  | Speedup (Num of threads: 1, 2, 4, 8, 16)
----------------------------|------------------------------------------
128,   24x24,  3, 64,   5x5 |2.18X, 2.13X, 1.73X, 1.64X, 1.66X
128,   24x24,  1, 64,   8x8 |2.00X, 1.98X, 1.93X, 1.91X, 1.91X
 32,   24x24,  3, 64,   5x5 |2.26X, 2.14X, 2.17X, 2.22X, 2.33X
128,   24x24,  3, 64,   3x3 |1.51X, 1.45X, 1.45X, 1.67X, 1.57X
 32,   14x14, 24, 64,   5x5 |1.21X, 1.19X, 1.16X, 1.70X, 1.17X
128, 128x128,  3, 96, 11x11 |2.17X, 2.18X, 2.19X, 2.20X, 2.18X

AVX2:

Parameters                  | Speedup (Num of threads: 1, 2, 4, 8, 16)
----------------------------|------------------------------------------
128,   24x24,  3, 64,   5x5 | 1.66X, 1.65X, 1.61X, 1.56X, 1.49X
 32,   24x24,  3, 64,   5x5 | 1.71X, 1.63X, 1.77X, 1.58X, 1.68X
128,   24x24,  1, 64,   5x5 | 1.44X, 1.40X, 1.38X, 1.37X, 1.33X
128,   24x24,  3, 64,   3x3 | 1.68X, 1.63X, 1.58X, 1.56X, 1.62X
128, 128x128,  3, 96, 11x11 | 1.36X, 1.36X, 1.37X, 1.37X, 1.37X

In the higher level benchmark cifar10, we observe a runtime improvement
of around 6% for AVX512 on Intel Skylake server (8 cores).

On lower level PackRhs micro-benchmarks specified in TensorFlow
tensorflow/core/kernels/eigen_spatial_convolutions_test.cc, we observe
the following runtime numbers:

AVX512:

Parameters                                                     | Runtime without patch (ns) | Runtime with patch (ns) | Speedup
---------------------------------------------------------------|----------------------------|-------------------------|---------
BM_RHS_NAME(PackRhs, 128, 24, 24, 3, 64, 5, 5, 1, 1, 256, 56)  |  41350                     | 15073                   | 2.74X
BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 1, 1, 256, 56)  |   7277                     |  7341                   | 0.99X
BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 2, 2, 256, 56)  |   8675                     |  8681                   | 1.00X
BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 1, 1, 256, 56)  |  24155                     | 16079                   | 1.50X
BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 2, 2, 256, 56)  |  25052                     | 17152                   | 1.46X
BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 1, 1, 256, 56) |  18269                     | 18345                   | 1.00X
BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 2, 4, 256, 56) |  19468                     | 19872                   | 0.98X
BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 1, 1, 36, 432)   | 156060                     | 42432                   | 3.68X
BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 2, 2, 36, 432)   | 132701                     | 36944                   | 3.59X

AVX2:

Parameters                                                     | Runtime without patch (ns) | Runtime with patch (ns) | Speedup
---------------------------------------------------------------|----------------------------|-------------------------|---------
BM_RHS_NAME(PackRhs, 128, 24, 24, 3, 64, 5, 5, 1, 1, 256, 56)  | 26233                      | 12393                   | 2.12X
BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 1, 1, 256, 56)  |  6091                      |  6062                   | 1.00X
BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 2, 2, 256, 56)  |  7427                      |  7408                   | 1.00X
BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 1, 1, 256, 56)  | 23453                      | 20826                   | 1.13X
BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 2, 2, 256, 56)  | 23167                      | 22091                   | 1.09X
BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 1, 1, 256, 56) | 23422                      | 23682                   | 0.99X
BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 2, 4, 256, 56) | 23165                      | 23663                   | 0.98X
BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 1, 1, 36, 432)   | 72689                      | 44969                   | 1.62X
BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 2, 2, 36, 432)   | 61732                      | 39779                   | 1.55X

All benchmarks on Intel Skylake server with 8 cores.
 2019-04-20 06:46:43 +00:00
Gustavo Lima Chaves
e763fcd09e Introducing "vectorized" byte on unpacket_traits structs 
This is a preparation to a change on gebp_traits, where a new template
argument will be introduced to dictate the packet size, so it won't be
bound to the current/max packet size only anymore.

By having packet types defined early on gebp_traits, one has now to
act on packet types, not scalars anymore, for the enum values defined
on that class. One approach for reaching the vectorizable/size
properties one needs there could be getting the packet's scalar again
with unpacket_traits<>, then the size/Vectorizable enum entries from
packet_traits<>. It turns out guards like "#ifndef
EIGEN_VECTORIZE_AVX512" at AVX/PacketMath.h will hide smaller packet
variations of packet_traits<> for some types (and it makes sense to
keep that). In other words, one can't go back to the scalar and create
a new PacketType, as this will always lead to the maximum packet type
for the architecture.

The less costly/invasive solution for that, thus, is to add the
vectorizable info on every unpacket_traits struct as well.
 2018-12-19 14:24:44 -08:00
Justin Carpentier
eabc7a4031 PR 465: Fix issue in RowMajor assignment in plain_matrix_type_row_major::type 
The type should be RowMajor
 2018-08-10 14:30:06 +02:00
Gael Guennebaud
1625476091 Add internall::is_identity compile-time helper 2018-07-11 14:00:24 +02:00
Gael Guennebaud
de9e31a06d Introduce the macro ei_declare_local_nested_eval to help allocating on the stack local temporaries via alloca, and let outer-products makes a good use of it. 
If successful, we should use it everywhere nested_eval is used to declare local dense temporaries.
 2018-07-09 15:41:14 +02:00
Andrea Bocci
f7124b3e46 Extend CUDA support to matrix inversion and selfadjointeigensolver 2018-06-11 18:33:24 +02:00
Gael Guennebaud
3ae2083e23 Make is_same_dense compatible with different scalar types. 2018-07-03 13:21:43 +02:00
Gael Guennebaud
725bd92903 fix stupid typo 2018-05-18 17:46:43 +02:00
Gael Guennebaud
a382bc9364 is_convertible<T,Index> does not seems to work well with MSVC 2013, so let's rather use __is_enum(T) for old MSVC versions 2018-05-18 17:02:27 +02:00
Gael Guennebaud
4dd767f455 add some internal checks 2018-05-18 13:59:55 +02:00
Christoph Hertzberg
0c9ad2f525 std::integral_constant is not C++03 compatible 2017-09-14 19:23:38 +02:00
Gael Guennebaud
6d42309f13 Fix compilation of Vector::operator()(enum) by treating enums as Index 2017-09-07 14:34:30 +02:00
Gael Guennebaud
25a1703579 Merged in ggael/eigen-flexidexing (pull request PR-294) 
generalized operator() for indexed access and slicing
 2017-01-26 08:04:23 +00:00
Gael Guennebaud
296d24be4d bug #1381: fix sparse.diagonal() used as a rvalue. 
The problem was that is "sparse" is not const, then sparse.diagonal() must have the
LValueBit flag meaning that sparse.diagonal().coeff(i) must returns a const reference,
const Scalar&. However, sparse::coeff() cannot returns a reference for a non-existing
zero coefficient. The trick is to return a reference to a local member of
evaluator<SparseMatrix>.
 2017-01-25 17:39:01 +01:00
Gael Guennebaud
c020d307a6 Make variable_if_dynamic<T> implicitely convertible to T 2017-01-11 17:08:05 +01:00
Gael Guennebaud
d32a43e33a Make sure that traits<CwiseBinaryOp>::Flags reports the correct storage order so that methods like .outerSize()/.innerSize() work properly. 2016-12-27 16:35:45 +01:00
Gael Guennebaud
6b8f637ab1 Harmless typo 2016-12-27 16:31:17 +01:00
Gael Guennebaud
c927af60ed Fix a performance regression in (mat*mat)*vec for which mat*mat was evaluated multiple times. 2016-11-30 17:59:13 +01:00
Gael Guennebaud
40f62974b7 bug #1328: workaround a compilation issue with gcc 4.2 2016-10-20 19:19:37 +02:00
Gael Guennebaud
ac5377e161 Improve cost estimation of complex division 2016-09-21 17:26:04 +02:00
Gael Guennebaud
228ae29591 Fix compilation on 32 bits systems. 2016-09-09 22:34:38 +02:00
Gael Guennebaud
471eac5399 bug #1195: move NumTraits::Div<>::Cost to internal::scalar_div_cost (with some specializations in arch/SSE and arch/AVX) 2016-09-08 08:36:27 +02:00
Gael Guennebaud
85fb517eaf Generalize ScalarBinaryOpTraits to any complex-real combination as defined by NumTraits (instead of supporting std::complex only). 2016-09-06 17:23:15 +02:00
Gael Guennebaud
218c37beb4 bug #1286: automatically detect the available prototypes of functors passed to CwiseNullaryExpr such that functors have only to implement the operators that matters among: 
operator()()
 operator()(i)
 operator()(i,j)
Linear access is also automatically detected based on the availability of operator()(i,j).
 2016-08-31 15:45:25 +02:00
Gael Guennebaud
4b4c296d6e Simplify ScalarBinaryOpTraits by removing the Defined enum, and extend its documentation. 2016-07-20 09:56:39 +02:00
Gael Guennebaud
79574e384e Make scalar_product_op the default (instead of void) 2016-07-18 12:03:05 +02:00
Gael Guennebaud
c50c73cae2 Fix missing specialization. 2016-06-24 23:10:39 +02:00
Gael Guennebaud
cd577a275c Relax promote_scalar_arg logic to enable promotion to Expr::Scalar if conversion to Expr::Literal fails. 
This is useful to cancel expression template at the scalar level, e.g. with AutoDiff<AutoDiff<>>.
This patch also defers calls to NumTraits in cases for which types are not directly compatible.
 2016-06-24 11:28:54 +02:00
Gael Guennebaud
76faf4a965 Introduce a NumTraits<T>::Literal type to be used for literals, and 
improve mixing type support in operations between arrays and scalars:
 - 2 * ArrayXcf is now optimized in the sense that the integer 2 is properly promoted to a float instead of a complex<float> (fix a regression)
 - 2.1 * ArrayXi is now forbiden (previously, 2.1 was converted to 2)
 - This mechanism should be applicable to any custom scalar type, assuming NumTraits<T>::Literal is properly defined (it defaults to T)
 2016-06-23 14:27:20 +02:00
Gael Guennebaud
64fcfd314f Implement scalar multiples and division by a scalar as a binary-expression with a constant expression. 
This slightly complexifies the type of the expressions and implies that we now have to distinguish between scalar*expr and expr*scalar to catch scalar-multiple expression (e.g., see BlasUtil.h), but this brings several advantages:
- it makes it clear on each side the scalar is applied,
- it clearly reflects that we are dealing with a binary-expression,
- the complexity of the type is hidden through macros defined at the end of Macros.h,
- distinguishing between "scalar op expr" and "expr op scalar" is important to support non commutative fields (like quaternions)
- "scalar op expr" is now fully equivalent to "ConstantExpr(scalar) op expr"
- scalar_multiple_op, scalar_quotient1_op and scalar_quotient2_op are not used anymore in officially supported modules (still used in Tensor)
 2016-06-14 11:26:57 +02:00
Gael Guennebaud
2c462f4201 Clean handling for void type in EIGEN_CHECK_BINARY_COMPATIBILIY 2016-06-06 23:11:38 +02:00
Gael Guennebaud
66e99ab6a1 Relax mixing-type constraints for binary coefficient-wise operators: 
- Replace internal::scalar_product_traits<A,B> by Eigen::ScalarBinaryOpTraits<A,B,OP>
- Remove the "functor_is_product_like" helper (was pretty ugly)
- Currently, OP is not used, but it is available to the user for fine grained tuning
- Currently, only the following operators have been generalized: *,/,+,-,=,*=,/=,+=,-=
- TODO: generalize all other binray operators (comparisons,pow,etc.)
- TODO: handle "scalar op array" operators (currently only * is handled)
- TODO: move the handling of the "void" scalar type to ScalarBinaryOpTraits
 2016-06-06 15:11:41 +02:00
Gael Guennebaud
53feb73b45 Remove dead code. 2016-06-02 22:19:55 +02:00
Gael Guennebaud
2c00ac0b53 Implement generic scalar*expr and expr*scalar operator based on scalar_product_traits. 
This is especially useful for custom scalar types, e.g., to enable float*expr<multi_prec> without conversion.
 2016-06-02 22:16:37 +02:00
Gael Guennebaud
1d3b253329 bug #1181: help MSVC inlining. 2016-05-31 17:23:42 +02:00
Gael Guennebaud
8ed1553d20 bug #632: implement general coefficient-wise "dense op sparse" operations through specialized evaluators instead of using SparseView. 
This permits to deal with arbitrary storage order, and to by-pass the more complex iterator of the sparse-sparse case.
 2016-01-30 14:39:50 +01:00
Gael Guennebaud
c1d900af61 bug #178: remove additional const on nested expression, and remove several const_cast. 2016-01-28 21:43:20 +01:00
Gael Guennebaud
8b0d1eb0f7 Fix numerous doxygen shortcomings, and workaround some clang -Wdocumentation warnings 2016-01-01 21:45:06 +01:00
Gael Guennebaud
addb7066e8 Workaround "empty paragraph" warning with clang -Wdocumentation 2015-12-30 16:45:44 +01:00
Gael Guennebaud
7a5f83ca60 Add overloads for real times sparse<complex> operations. 
This avoids real to complex conversions, and also fixes a compilation issue with MSVC.
 2015-10-29 03:55:39 -07:00
Gael Guennebaud
77ff3386b7 Refactoring of the cost model: 
- Dynamic is now an invalid value
 - introduce a HugeCost constant to be used for runtime-cost values or arbitrarily huge cost
 - add sanity checks for cost values: must be >=0 and not too large
This change provides several benefits:
 - it fixes shortcoming is some cost computation where the Dynamic case was not properly handled.
 - it simplifies cost computation logic, and should avoid future similar shortcomings.
 - it allows to distinguish between different level of dynamic/huge/infinite cost
 - it should enable further simplifications in the computation of costs (save compilation time)
 2015-10-28 11:42:14 +01:00
Gael Guennebaud
0fc8954282 Improve readibility of EIGEN_DEBUG_ASSIGN mode. 2015-10-27 10:38:49 +01:00
Gael Guennebaud
2598f3987e Add a plain_object_eval<> helper returning a plain object type based on evaluator's Flags, 
and base nested_eval on it.
 2015-10-14 10:12:58 +02:00
Gael Guennebaud
a4cc4c1e5e Clarify note in nested_eval for evaluator creating temporaries. 2015-10-09 14:57:51 +02:00
Gael Guennebaud
26cde4db3c Define Permutation*<>::Scalar to 'void', re-enable scalar type compatibility check in assignment while relaxing this test for void types. 2015-10-06 17:18:06 +02:00
Christoph Hertzberg
5ad7981f73 Use full packet size for Dynamic-sized objects (otherwise, the unalignedcount unit test fails with AVX enabled) 2015-09-02 22:51:43 +02:00
Gael Guennebaud
febcce34f1 Enable vectorization with half-packets 2015-08-07 20:05:31 +02:00
Gael Guennebaud
ce57dbd937 Let unpacket_traits<> exposes the required alignment and make use of it everywhere 2015-08-07 10:44:01 +02:00