ets
Built-In Term Storage
This module is an interface to the Erlang built-in term storage
BIFs. These provide the ability to store very large quantities of
data in an Erlang runtime system, and to have constant access
time to the data. (In the case of ordered_set
, see below,
access time is proportional to the logarithm of the number of
objects stored).
Data is organized as a set of dynamic tables, which can store tuples. Each table is created by a process. When the process terminates, the table is automatically destroyed. Every table has access rights set at creation.
Tables are divided into four different types, set
,
ordered_set
, bag
and duplicate_bag
.
A set
or ordered_set
table can only have one object
associated with each key. A bag
or duplicate_bag
can
have many objects associated with each key.
The number of tables stored at one Erlang node is limited.
The current default limit is approximately 1400 tables. The upper
limit can be increased by setting the environment variable
ERL_MAX_ETS_TABLES
before starting the Erlang runtime
system (i.e. with the -env
option to
erl
/werl
). The actual limit may be slightly higher
than the one specified, but never lower.
Note that there is no automatic garbage collection for tables.
Even if there are no references to a table from any process, it
will not automatically be destroyed unless the owner process
terminates. It can be destroyed explicitly by using
delete/1
. The default owner is the process that created the
table. Table ownership can be transferred at process termination
by using the heir option or explicitly
by calling give_away/3.
Some implementation details:
- In the current implementation, every object insert and look-up operation results in a copy of the object.
'$end_of_table'
should not be used as a key since this atom is used to mark the end of the table when usingfirst
/next
.
Also worth noting is the subtle difference between
matching and comparing equal, which is
demonstrated by the different table types set
and
ordered_set
. Two Erlang terms match
if they are of
the same type and have the same value, so that 1
matches
1
, but not 1.0
(as 1.0
is a float()
and not an integer()
). Two Erlang terms compare equal if they either are of the same type and value, or if
both are numeric types and extend to the same value, so that
1
compares equal to both 1
and 1.0
. The
ordered_set
works on the Erlang term order and
there is no defined order between an integer()
and a
float()
that extends to the same value, hence the key
1
and the key 1.0
are regarded as equal in an
ordered_set
table.
Failure
In general, the functions below will exit with reason
badarg
if any argument is of the wrong format, if the
table identifier is invalid or if the operation is denied due to
table access rights (protected
or private).
Concurrency
This module provides some limited support for concurrent access. All updates to single objects are guaranteed to be both atomic and isolated. This means that an updating operation towards a single object will either succeed or fail completely without any effect at all (atomicy). Nor can any intermediate results of the update be seen by other processes (isolation). Some functions that update several objects state that they even guarantee atomicy and isolation for the entire operation. In database terms the isolation level can be seen as "serializable", as if all isolated operations were carried out serially, one after the other in a strict order.
No other support is available within ETS that would guarantee
consistency between objects. However, the safe_fixtable/2
function can be used to guarantee that a sequence of
first/1
and next/2
calls will traverse the table
without errors and that each existing object in the table is visited
exactly once, even if another process (or the same process)
simultaneously deletes or inserts objects into the table.
Nothing more is guaranteed; in particular objects that are inserted
or deleted during such a traversal may be visited once or not at all.
Functions that internally traverse over a table, like select
and match
, will give the same guarantee as safe_fixtable
.
Match Specifications
Some of the functions uses a match specification, match_spec. A brief explanation is given in select/2. For a detailed description, see the chapter "Match specifications in Erlang" in ERTS User's Guide.
Opaque continuation used by
select/1,3
,
select_reverse/1,3
,
match/1,3
, and
match_object/1,3
.
A match specification, see above.
A compiled match specification.
A table identifier, as returned by new/2.
Functions
all/0
Returns a list of all tables at the node. Named tables are given by their names, unnamed tables are given by their table identifiers.
delete/1
Deletes the entire table
.
delete/2
Deletes all objects with the key
from the table
.
delete_all_objects/1
Delete all objects in the ETS table
.
The operation is guaranteed to be
atomic and isolated.
delete_object/2
Delete the exact object
from the ETS table,
leaving objects with the same key but other differences
(useful for type bag
). In a duplicate_bag
, all
instances of the object will be deleted.
file2tab/1
Reads a file produced by tab2file/2 or
tab2file/3 and creates the
corresponding table
.
Equivalent to file2tab(
.
file2tab/2
Reads a file produced by tab2file/2 or
tab2file/3 and creates the
corresponding table
.
The currently only supported option is {verify,boolean()}
. If
verification is turned on (by means of specifying
{verify,true}
), the function utilizes whatever
information is present in the file to assert that the
information is not damaged. How this is done depends on which
extended_info
was written using
tab2file/3.
If no extended_info
is present in the file and
{verify,true}
is specified, the number of objects
written is compared to the size of the original table when the
dump was started. This might make verification fail if the
table was
public
and objects were added or removed while the
table was dumped to file. To avoid this type of problems,
either do not verify files dumped while updated simultaneously
or use the {extended_info, [object_count]}
option to
tab2file/3, which
extends the information in the file with the number of objects
actually written.
If verification is turned on and the file was written with
the option {extended_info, [md5sum]}
, reading the file
is slower and consumes radically more CPU time than
otherwise.
{verify,false}
is the default.
first/1
Returns the first key
in the table
.
If the table is of the ordered_set
type, the first key
in Erlang term order will be returned. If the table is of any
other type, the first key according to the table's internal
order will be returned. If the table is empty,
'$end_of_table'
will be returned.
Use next/2
to find subsequent keys in the table.
foldl/3
is returned if the table is empty.
This function is similar to lists:foldl/3
. The order in
which the elements of the table are traversed is unspecified,
except for tables of type ordered_set
, for which they
are traversed first to last.
If
inserts objects into the table, or another
process inserts objects into the table, those objects may
(depending on key ordering) be included in the traversal.
foldr/3
is returned if the table is empty.
This function is similar to lists:foldr/3
. The order in
which the elements of the table are traversed is unspecified,
except for tables of type ordered_set
, for which they
are traversed last to first.
If
inserts objects into the table, or another
process inserts objects into the table, those objects may
(depending on key ordering) be included in the traversal.
from_dets/2
Fills an already created ETS table with the objects in the
already opened Dets table named
. The existing
objects of the ETS table are kept unless overwritten.
Throws a badarg error if any of the tables does not exist or the dets table is not open.
fun2ms/1
Pseudo function that by means of a parse_transform
translates
typed as parameter in the
function call to a
match_spec. With
"literal" is meant that the fun needs to textually be written
as the parameter of the function, it cannot be held in a
variable which in turn is passed to the function).
The parse transform is implemented in the module
ms_transform
and the source must include the
file ms_transform.hrl
in STDLIB for this
pseudo function to work. Failing to include the hrl file in
the source will result in a runtime error, not a compile
time ditto. The include file is easiest included by adding
the line
-include_lib("stdlib/include/ms_transform.hrl").
to
the source file.
The fun is very restricted, it can take only a single
parameter (the object to match): a sole variable or a
tuple. It needs to use the is_
guard tests.
Language constructs that have no representation
in a match_spec (like if
, case
, receive
etc) are not allowed.
The return value is the resulting match_spec.
Example:
1> ets:fun2ms(fun({M,N}) when N > 3 -> M end).
[{{'$1','$2'},[{'>','$2',3}],['$1']}]
Variables from the environment can be imported, so that this works:
2>X=3.
3 3>ets:fun2ms(fun({M,N}) when N > X -> M end).
[{{'$1','$2'},[{'>','$2',{const,3}}],['$1']}]
The imported variables will be replaced by match_spec
const
expressions, which is consistent with the
static scoping for Erlang funs. Local or global function
calls can not be in the guard or body of the fun however.
Calls to builtin match_spec functions of course is allowed:
4>ets:fun2ms(fun({M,N}) when N > X, is_atomm(M) -> M end).
Error: fun containing local Erlang function calls ('is_atomm' called in guard) cannot be translated into match_spec {error,transform_error} 5>ets:fun2ms(fun({M,N}) when N > X, is_atom(M) -> M end).
[{{'$1','$2'},[{'>','$2',{const,3}},{is_atom,'$1'}],['$1']}]
As can be seen by the example, the function can be called from the shell too. The fun needs to be literally in the call when used from the shell as well. Other means than the parse_transform are used in the shell case, but more or less the same restrictions apply (the exception being records, as they are not handled by the shell).
Warning!
If the parse_transform is not applied to a module which
calls this pseudo function, the call will fail in runtime
(with a badarg
). The module ets
actually
exports a function with this name, but it should never
really be called except for when using the function in the
shell. If the parse_transform
is properly applied by
including the ms_transform.hrl
header file, compiled
code will never call the function, but the function call is
replaced by a literal match_spec.
For more information, see ms_transform(3).
give_away/3
Make process
the new owner of table
.
If successful, the message
{'ETS-TRANSFER',
will be sent
to the new owner.
The process
must be alive, local and not already the
owner of the table. The calling process must be the table owner.
Note that give_away
does not at all affect the
heir option of the table. A table
owner can for example set the heir
to itself, give the table
away and then get it back in case the receiver terminates.
i/0
Displays information about all ETS tables on tty.
i/1
Browses the table
on tty.
info/1
Returns information about the table
as a list of
tuples. If
has the correct type
for a table identifier, but does not refer to an existing ETS
table, undefined
is returned. If
is not of the
correct type, this function fails with reason badarg
.
{compressed, boolean()}
Indicates if the table is compressed or not.{heir, pid() | none}
The pid of the heir of the table, ornone
if no heir is set.{keypos, integer() >= 1}
The key position.{memory, integer() >= 0
The number of words allocated to the table.{name, atom()}
The name of the table.{named_table, boolean()}
Indicates if the table is named or not.{node, node()}
The node where the table is stored. This field is no longer meaningful as tables cannot be accessed from other nodes.{owner, pid()}
The pid of the owner of the table.{protection, access()}
The table access rights.{size, integer() >= 0
The number of objects inserted in the table.{type, type()}
The table type.
info/2
Returns the information associated with Item
for
the table
, or returns undefined
if Tab
does not refer an existing ETS table.
If
is not of the correct type, or if
is not
one of the allowed values, this function fails with reason badarg
.
Warning!
In R11B and earlier, this function would not fail but return
undefined
for invalid values for Item
.
In addition to the {
pairs defined for info/1
, the following items are
allowed:
Item=fixed, Value=boolean()
Indicates if the table is fixed by any process or not.-
Item=safe_fixed, Value={FirstFixed,Info}|false
If the table has been fixed using
safe_fixtable/2
, the call returns a tuple whereFirstFixed
is the time when the table was first fixed by a process, which may or may not be one of the processes it is fixed by right now.Info
is a possibly empty lists of tuples{Pid,RefCount}
, one tuple for every process the table is fixed by right now.RefCount
is the value of the reference counter, keeping track of how many times the table has been fixed by the process.If the table never has been fixed, the call returns
false
. Item=stats, Value=tuple()
Returns internal statistics about set, bag and duplicate_bag tables on an internal format used by OTP test suites. Not for production use.
init_table/2
Replaces the existing objects of the table
with
objects created by calling the input function
,
see below. This function is provided for compatibility with
the dets
module, it is not more efficient than filling
a table by using ets:insert/2
.
When called with the argument read
the function
is assumed to return end_of_input
when
there is no more input, or {
, where
is a list of objects and Fun
is a new
input function. Any other value Value is returned as an error
{error, {init_fun, Value}}
. Each input function will be
called exactly once, and should an error occur, the last
function is called with the argument close
, the reply
of which is ignored.
If the type of the table is set
and there is more
than one object with a given key, one of the objects is
chosen. This is not necessarily the last object with the given
key in the sequence of objects returned by the input
functions. This holds also for duplicated
objects stored in tables of type bag
.
insert/2
Inserts the object or all of the objects in the list
into the table
.
If the table is a set
and the key of the inserted
objects matches the key of any object in the table,
the old object will be replaced. If the table is an
ordered_set
and the key of the inserted object
compares equal to the key of any object in the
table, the old object is also replaced. If the list contains
more than one object with matching keys and the table is a
set
, one will be inserted, which one is
not defined. The same thing holds for ordered_set
, but
will also happen if the keys compare equal.
The entire operation is guaranteed to be atomic and isolated, even when a list of objects is inserted.
insert_new/2
This function works exactly like insert/2
, with the
exception that instead of overwriting objects with the same
key (in the case of set
or ordered_set
) or
adding more objects with keys already existing in the table
(in the case of bag
and duplicate_bag
), it
simply returns false
. If
is a
list, the function checks every key prior to
inserting anything. Nothing will be inserted if not
all keys present in the list are absent from the
table. Like insert/2
, the entire operation is guaranteed to be
atomic and isolated.
is_compiled_ms/1
This function is used to check if a term is a valid
compiled match_spec.
The compiled match_spec is an opaque datatype which can
not be sent between Erlang nodes nor be stored on
disk. Any attempt to create an external representation of a
compiled match_spec will result in an empty binary
(<<>>
). As an example, the following
expression:
ets:is_compiled_ms(ets:match_spec_compile([{'_',[],[true]}])).
will yield true
, while the following expressions:
MS = ets:match_spec_compile([{'_',[],[true]}]), Broken = binary_to_term(term_to_binary(MS)), ets:is_compiled_ms(Broken).
will yield false, as the variable Broken
will contain
a compiled match_spec that has passed through external
representation.
Note!
The fact that compiled match_specs has no external representation is for performance reasons. It may be subject to change in future releases, while this interface will still remain for backward compatibility reasons.
last/1
Returns the last key
according to Erlang term
order in the table Tab
of the ordered_set
type.
If the table is of any other type, the function is synonymous
to first/2
. If the table is empty,
'$end_of_table'
is returned.
Use prev/2
to find preceding keys in the table.
lookup/2
Returns a list of all objects with the key
in
the table
.
In the case of set, bag and duplicate_bag
, an object
is returned only if the given key matches the key
of the object in the table. If the table is an
ordered_set
however, an object is returned if the key
given compares equal to the key of an object in the
table. The difference being the same as between =:=
and ==
. As an example, one might insert an object
with the
integer()
1
as a key in an ordered_set
and get the object returned as a result of doing a
lookup/2
with the float()
1.0
as the
key to search for.
If the table is of type set
or ordered_set
,
the function returns either the empty list or a list with one
element, as there cannot be more than one object with the same
key. If the table is of type bag
or
duplicate_bag
, the function returns a list of
arbitrary length.
Note that the time order of object insertions is preserved; the first object inserted with the given key will be first in the resulting list, and so on.
Insert and look-up times in tables of type set
,
bag
and duplicate_bag
are constant, regardless
of the size of the table. For the ordered_set
data-type, time is proportional to the (binary) logarithm of
the number of objects.
lookup_element/3
If the table
is of type set
or
ordered_set
, the function returns the
:th
element of the object with the key
.
If the table is of type bag
or duplicate_bag
,
the functions returns a list with the
:th element of
every object with the key
.
If no object with the key
exists, the function
will exit with reason badarg
.
The difference between set
, bag
and
duplicate_bag
on one hand, and ordered_set
on
the other, regarding the fact that ordered_set
's
view keys as equal when they compare equal
whereas the other table types only regard them equal when
they match, naturally holds for
lookup_element
as well as for lookup
.
match/2
Matches the objects in the table
against the
pattern
.
A pattern is a term that may contain:
- bound parts (Erlang terms),
'_'
which matches any Erlang term, and- pattern variables:
'$N'
whereN
=0,1,...
The function returns a list with one element for each matching object, where each element is an ordered list of pattern variable bindings. An example:
6>ets:match(T, '$1').
% Matches every object in the table [[{rufsen,dog,7}],[{brunte,horse,5}],[{ludde,dog,5}]] 7>ets:match(T, {'_',dog,'$1'}).
[[7],[5]] 8>ets:match(T, {'_',cow,'$1'}).
[]
If the key is specified in the pattern, the match is very efficient. If the key is not specified, i.e. if it is a variable or an underscore, the entire table must be searched. The search time can be substantial if the table is very large.
On tables of the ordered_set
type, the result is in
the same order as in a first/next
traversal.
match/3
Works like ets:match/2
but only returns a limited
(
) number of matching objects. The
term can then be used in subsequent calls
to ets:match/1
to get the next chunk of matching
objects. This is a space efficient way to work on objects in a
table which is still faster than traversing the table object
by object using ets:first/1
and ets:next/1
.
'$end_of_table'
is returned if the table is empty.
match/1
Continues a match started with ets:match/3
. The next
chunk of the size given in the initial ets:match/3
call is returned together with a new
that can be used in subsequent calls to this function.
'$end_of_table'
is returned when there are no more
objects in the table.
match_delete/2
Deletes all objects which match the pattern
from the table
. See match/2
for a
description of patterns.
match_object/2
Matches the objects in the table
against the
pattern
. See match/2
for a description
of patterns. The function returns a list of all objects which
match the pattern.
If the key is specified in the pattern, the match is very efficient. If the key is not specified, i.e. if it is a variable or an underscore, the entire table must be searched. The search time can be substantial if the table is very large.
On tables of the ordered_set
type, the result is in
the same order as in a first/next
traversal.
match_object/3
Works like ets:match_object/2
but only returns a
limited (
) number of matching objects. The
term can then be used in subsequent calls
to ets:match_object/1
to get the next chunk of matching
objects. This is a space efficient way to work on objects in a
table which is still faster than traversing the table object
by object using ets:first/1
and ets:next/1
.
'$end_of_table'
is returned if the table is empty.
match_object/1
Continues a match started with ets:match_object/3
.
The next chunk of the size given in the initial
ets:match_object/3
call is returned together with a
new
that can be used in subsequent calls
to this function.
'$end_of_table'
is returned when there are no more
objects in the table.
match_spec_compile/1
This function transforms a
match_spec into an
internal representation that can be used in subsequent calls
to ets:match_spec_run/2
. The internal representation is
opaque and can not be converted to external term format and
then back again without losing its properties (meaning it can
not be sent to a process on another node and still remain a
valid compiled match_spec, nor can it be stored on disk).
The validity of a compiled match_spec can be checked using
ets:is_compiled_ms/1
.
If the term
can not be compiled (does not
represent a valid match_spec), a badarg
fault is
thrown.
Note!
This function has limited use in normal code, it is used by
Dets to perform the dets:select
operations.
match_spec_run/2
This function executes the matching specified in a
compiled match_spec on
a list of tuples. The
term should be
the result of a call to ets:match_spec_compile/1
and
is hence the internal representation of the match_spec one
wants to use.
The matching will be executed on each element in
and the function returns a list containing all results. If an
element in
does not match, nothing is returned
for that element. The length of the result list is therefore
equal or less than the the length of the parameter
. The two calls in the following example will give
the same result (but certainly not the same execution
time...):
Table = ets:new... MatchSpec = .... % The following call... ets:match_spec_run(ets:tab2list(Table), ets:match_spec_compile(MatchSpec)), % ...will give the same result as the more common (and more efficient) ets:select(Table,MatchSpec),
Note!
This function has limited use in normal code, it is used by
Dets to perform the dets:select
operations and by
Mnesia during transactions.
member/2
Works like lookup/2
, but does not return the objects.
The function returns true
if one or more elements in
the table has the key
, false
otherwise.
new/2
Creates a new table and returns a table identifier which can be used in subsequent operations. The table identifier can be sent to other processes so that a table can be shared between different processes within a node.
The parameter
is a list of atoms which
specifies table type, access rights, key position and if the
table is named or not. If one or more options are left out,
the default values are used. This means that not specifying
any options ([]
) is the same as specifying
[set, protected, {keypos,1}, {heir,none}, {write_concurrency,false}, {read_concurrency,false}]
.
-
set
The table is aset
table - one key, one object, no order among objects. This is the default table type. -
ordered_set
The table is aordered_set
table - one key, one object, ordered in Erlang term order, which is the order implied by the < and > operators. Tables of this type have a somewhat different behavior in some situations than tables of the other types. Most notably theordered_set
tables regard keys as equal when they compare equal, not only when they match. This means that to anordered_set
, theinteger()
1
and thefloat()
1.0
are regarded as equal. This also means that the key used to lookup an element not necessarily matches the key in the elements returned, iffloat()
's andinteger()
's are mixed in keys of a table. -
bag
The table is abag
table which can have many objects, but only one instance of each object, per key. -
duplicate_bag
The table is aduplicate_bag
table which can have many objects, including multiple copies of the same object, per key. -
public
Any process may read or write to the table. -
protected
The owner process can read and write to the table. Other processes can only read the table. This is the default setting for the access rights. -
private
Only the owner process can read or write to the table. -
named_table
If this option is present, the name
is associated with the table identifier. The name can then be used instead of the table identifier in subsequent operations.Name -
{keypos,
Specfies which element in the stored tuples should be used as key. By default, it is the first element, i.e.Pos }
. However, this is not always appropriate. In particular, we do not want the first element to be the key if we want to store Erlang records in a table.Pos =1Note that any tuple stored in the table must have at least
number of elements.Pos -
{heir,
Pid ,HeirData } | {heir,none}
Set a process as heir. The heir will inherit the table if the owner terminates. The message{'ETS-TRANSFER',tid(),FromPid,
will be sent to the heir when that happens. The heir must be a local process. Default heir isHeirData }none
, which will destroy the table when the owner terminates. -
{write_concurrency,boolean()}
Performance tuning. Default isfalse
, in which case an operation that mutates (writes to) the table will obtain exclusive access, blocking any concurrent access of the same table until finished. If set totrue
, the table is optimized towards concurrent write access. Different objects of the same table can be mutated (and read) by concurrent processes. This is achieved to some degree at the expense of memory consumption and the performance of sequential access and concurrent reading. Thewrite_concurrency
option can be combined with the read_concurrency option. You typically want to combine these when large concurrent read bursts and large concurrent write bursts are common (see the documentation of the read_concurrency option for more information). Note that this option does not change any guarantees about atomicy and isolation. Functions that makes such promises over several objects (likeinsert/2
) will gain less (or nothing) from this option.In current implementation, table type
ordered_set
is not affected by this option. Also, the memory consumption inflicted by bothwrite_concurrency
andread_concurrency
is a constant overhead per table. This overhead can be especially large when both options are combined. -
{read_concurrency,boolean()}
Performance tuning. Default isfalse
. When set totrue
, the table is optimized for concurrent read operations. When this option is enabled on a runtime system with SMP support, read operations become much cheaper; especially on systems with multiple physical processors. However, switching between read and write operations becomes more expensive. You typically want to enable this option when concurrent read operations are much more frequent than write operations, or when concurrent reads and writes comes in large read and write bursts (i.e., lots of reads not interrupted by writes, and lots of writes not interrupted by reads). You typically do not want to enable this option when the common access pattern is a few read operations interleaved with a few write operations repeatedly. In this case you will get a performance degradation by enabling this option. Theread_concurrency
option can be combined with the write_concurrency option. You typically want to combine these when large concurrent read bursts and large concurrent write bursts are common. -
compressed
If this option is present, the table data will be stored in a more compact format to consume less memory. The downside is that it will make table operations slower. Especially operations that need to inspect entire objects, such asmatch
andselect
, will get much slower. The key element is not compressed in current implementation.
next/2
Returns the next key
, following the key
in the table
. If the table is of the
ordered_set
type, the next key in Erlang term order is
returned. If the table is of any other type, the next key
according to the table's internal order is returned. If there
is no next key, '$end_of_table'
is returned.
Use first/1
to find the first key in the table.
Unless a table of type set
, bag
or
duplicate_bag
is protected using
safe_fixtable/2
, see below, a traversal may fail if
concurrent updates are made to the table. If the table is of
type ordered_set
, the function returns the next key in
order, even if the object does no longer exist.
prev/2
Returns the previous key
, preceding the key
according the Erlang term order in the table
of the ordered_set
type. If the table is of
any other type, the function is synonymous to next/2
.
If there is no previous key, '$end_of_table'
is
returned.
Use last/1
to find the last key in the table.
rename/2
Renames the named table
to the new name
. Afterwards, the old name can not be used to
access the table. Renaming an unnamed table has no effect.
repair_continuation/2
This function can be used to restore an opaque continuation
returned by ets:select/3
or ets:select/1
if the
continuation has passed through external term format (been
sent between nodes or stored on disk).
The reason for this function is that continuation terms
contain compiled match_specs and therefore will be
invalidated if converted to external term format. Given that
the original match_spec is kept intact, the continuation can
be restored, meaning it can once again be used in subsequent
ets:select/1
calls even though it has been stored on
disk or on another node.
As an example, the following sequence of calls will fail:
T=ets:new(x,[]), ... {_,C} = ets:select(T,ets:fun2ms(fun({N,_}=A) when (N rem 10) =:= 0 -> A end),10), Broken = binary_to_term(term_to_binary(C)), ets:select(Broken).
...while the following sequence will work:
T=ets:new(x,[]), ... MS = ets:fun2ms(fun({N,_}=A) when (N rem 10) =:= 0 -> A end), {_,C} = ets:select(T,MS,10), Broken = binary_to_term(term_to_binary(C)), ets:select(ets:repair_continuation(Broken,MS)).
...as the call to ets:repair_continuation/2
will
reestablish the (deliberately) invalidated continuation
Broken
.
Note!
This function is very rarely needed in application code. It
is used by Mnesia to implement distributed select/3
and select/1
sequences. A normal application would
either use Mnesia or keep the continuation from being
converted to external format.
The reason for not having an external representation of a compiled match_spec is performance. It may be subject to change in future releases, while this interface will remain for backward compatibility.
safe_fixtable/2
Fixes a table of the set
, bag
or
duplicate_bag
table type for safe traversal.
A process fixes a table by calling
safe_fixtable(
. The table remains fixed until
the process releases it by calling
safe_fixtable(
, or until the process
terminates.
If several processes fix a table, the table will remain fixed until all processes have released it (or terminated). A reference counter is kept on a per process basis, and N consecutive fixes requires N releases to actually release the table.
When a table is fixed, a sequence of first/1
and
next/2
calls are guaranteed to succeed and each object in
the table will only be returned once, even if objects
are removed or inserted during the traversal.
The keys for new objects inserted during the traversal may
be returned by next/2
(it depends on the internal ordering of the keys). An example:
clean_all_with_value(Tab,X) -> safe_fixtable(Tab,true), clean_all_with_value(Tab,X,ets:first(Tab)), safe_fixtable(Tab,false). clean_all_with_value(Tab,X,'$end_of_table') -> true; clean_all_with_value(Tab,X,Key) -> case ets:lookup(Tab,Key) of [{Key,X}] -> ets:delete(Tab,Key); _ -> true end, clean_all_with_value(Tab,X,ets:next(Tab,Key)).
Note that no deleted objects are actually removed from a fixed table until it has been released. If a process fixes a table but never releases it, the memory used by the deleted objects will never be freed. The performance of operations on the table will also degrade significantly.
Use info/2
to retrieve information about which
processes have fixed which tables. A system with a lot of
processes fixing tables may need a monitor which sends alarms
when tables have been fixed for too long.
Note that for tables of the ordered_set
type,
safe_fixtable/2
is not necessary as calls to
first/1
and next/2
will always succeed.
select/2
Matches the objects in the table
using a
match_spec. This is a
more general call than the ets:match/2
and
ets:match_object/2
calls. In its simplest forms the
match_specs look like this:
- MatchSpec = [MatchFunction]
- MatchFunction = {MatchHead, [Guard], [Result]}
- MatchHead = "Pattern as in ets:match"
- Guard = {"Guardtest name", ...}
- Result = "Term construct"
This means that the match_spec is always a list of one or
more tuples (of arity 3). The tuples first element should be
a pattern as described in the documentation of
ets:match/2
. The second element of the tuple should
be a list of 0 or more guard tests (described below). The
third element of the tuple should be a list containing a
description of the value to actually return. In almost all
normal cases the list contains exactly one term which fully
describes the value to return for each object.
The return value is constructed using the "match variables"
bound in the MatchHead or using the special match variables
'$_'
(the whole matching object) and '$$'
(all
match variables in a list), so that the following
ets:match/2
expression:
ets:match(Tab,{'$1','$2','$3'})
is exactly equivalent to:
ets:select(Tab,[{{'$1','$2','$3'},[],['$$']}])
- and the following ets:match_object/2
call:
ets:match_object(Tab,{'$1','$2','$1'})
is exactly equivalent to
ets:select(Tab,[{{'$1','$2','$1'},[],['$_']}])
Composite terms can be constructed in the Result
part
either by simply writing a list, so that this code:
ets:select(Tab,[{{'$1','$2','$3'},[],['$$']}])
gives the same output as:
ets:select(Tab,[{{'$1','$2','$3'},[],[['$1','$2','$3']]}])
i.e. all the bound variables in the match head as a list. If
tuples are to be constructed, one has to write a tuple of
arity 1 with the single element in the tuple being the tuple
one wants to construct (as an ordinary tuple could be mistaken
for a Guard
). Therefore the following call:
ets:select(Tab,[{{'$1','$2','$1'},[],['$_']}])
gives the same output as:
ets:select(Tab,[{{'$1','$2','$1'},[],[{{'$1','$2','$3'}}]}])
- this syntax is equivalent to the syntax used in the trace patterns (see dbg(3)).
The Guard
s are constructed as tuples where the first
element is the name of the test and the rest of the elements
are the parameters of the test. To check for a specific type
(say a list) of the element bound to the match variable
'$1'
, one would write the test as
{is_list, '$1'}
. If the test fails, the object in the
table will not match and the next MatchFunction
(if
any) will be tried. Most guard tests present in Erlang can be
used, but only the new versions prefixed is_
are
allowed (like is_float
, is_atom
etc).
The Guard
section can also contain logic and
arithmetic operations, which are written with the same syntax
as the guard tests (prefix notation), so that a guard test
written in Erlang looking like this:
is_integer(X), is_integer(Y), X + Y < 4711
is expressed like this (X replaced with '$1' and Y with '$2'):
[{is_integer, '$1'}, {is_integer, '$2'}, {'<', {'+', '$1', '$2'}, 4711}]
On tables of the ordered_set
type, objects are visited
in the same order as in a first/next
traversal. This means that the match specification will be
executed against objects with keys in the first/next
order and the corresponding result list will be in the order of that
execution.
select/3
Works like ets:select/2
but only returns a limited
(
) number of matching objects. The
term can then be used in subsequent calls
to ets:select/1
to get the next chunk of matching
objects. This is a space efficient way to work on objects in a
table which is still faster than traversing the table object
by object using ets:first/1
and ets:next/1
.
'$end_of_table'
is returned if the table is empty.
select/1
Continues a match started with
ets:select/3
. The next
chunk of the size given in the initial ets:select/3
call is returned together with a new
that can be used in subsequent calls to this function.
'$end_of_table'
is returned when there are no more
objects in the table.
select_count/2
Matches the objects in the table
using a
match_spec. If the
match_spec returns true
for an object, that object
considered a match and is counted. For any other result from
the match_spec the object is not considered a match and is
therefore not counted.
The function could be described as a match_delete/2
that does not actually delete any elements, but only counts
them.
The function returns the number of objects matched.
select_delete/2
Matches the objects in the table
using a
match_spec. If the
match_spec returns true
for an object, that object is
removed from the table. For any other result from the
match_spec the object is retained. This is a more general
call than the ets:match_delete/2
call.
The function returns the number of objects actually deleted from the table.
Note!
The match_spec
has to return the atom true
if
the object is to be deleted. No other return value will get the
object deleted, why one can not use the same match specification for
looking up elements as for deleting them.
select_reverse/2
Works like select/2
, but returns the list in reverse
order for the ordered_set
table type. For all other table
types, the return value is identical to that of select/2
.
select_reverse/3
Works like select/3
, but for the ordered_set
table type, traversing is done starting at the last object in
Erlang term order and moves towards the first. For all other
table types, the return value is identical to that of
select/3
.
Note that this is not equivalent to
reversing the result list of a select/3
call, as the result list
is not only reversed, but also contains the last
matching objects in the table, not the first.
select_reverse/1
Continues a match started with
ets:select_reverse/3
. If the table is an
ordered_set
, the traversal of the table will continue
towards objects with keys earlier in the Erlang term order. The
returned list will also contain objects with keys in reverse
order.
For all other table types, the behaviour is exactly that of select/1
.
Example:
1> T = ets:new(x,[ordered_set]).
2> [ ets:insert(T,{N}) || N <- lists:seq(1,10) ].
...
3> {R0,C0} = ets:select_reverse(T,[{'_',[],['$_']}],4).
...
4> R0.
[{10},{9},{8},{7}]
5> {R1,C1} = ets:select_reverse(C0).
...
6> R1.
[{6},{5},{4},{3}]
7> {R2,C2} = ets:select_reverse(C1).
...
8> R2.
[{2},{1}]
9> '$end_of_table' = ets:select_reverse(C2).
...
setopts/2
Set table options. The only option that currently is allowed to be set after the table has been created is heir. The calling process must be the table owner.
slot/2
This function is mostly for debugging purposes, Normally
one should use first/next
or last/prev
instead.
Returns all objects in the
:th slot of the table
. A table can be traversed by repeatedly calling
the function, starting with the first slot
and
ending when '$end_of_table'
is returned.
The function will fail with reason badarg
if the
argument is out of range.
Unless a table of type set
, bag
or
duplicate_bag
is protected using
safe_fixtable/2
, see above, a traversal may fail if
concurrent updates are made to the table. If the table is of
type ordered_set
, the function returns a list
containing the
:th object in Erlang term order.
tab2file/2
Dumps the table
to the file
.
Equivalent to tab2file(
tab2file/3
Dumps the table
to the file
.
When dumping the table, certain information about the table is dumped to a header at the beginning of the dump. This information contains data about the table type, name, protection, size, version and if it's a named table. It also contains notes about what extended information is added to the file, which can be a count of the objects in the file or a MD5 sum of the header and records in the file.
The size field in the header might not correspond to the actual number of records in the file if the table is public and records are added or removed from the table during dumping. Public tables updated during dump, and that one wants to verify when reading, needs at least one field of extended information for the read verification process to be reliable later.
The extended_info
option specifies what extra
information is written to the table dump:
object_count
The number of objects actually written to the file is noted in the file footer, why verification of file truncation is possible even if the file was updated during dump.
md5sum
The header and objects in the file are checksummed using the built in MD5 functions. The MD5 sum of all objects is written in the file footer, so that verification while reading will detect the slightest bitflip in the file data. Using this costs a fair amount of CPU time.
Whenever the extended_info
option is used, it
results in a file not readable by versions of ets prior to
that in stdlib-1.15.1
tab2list/1
Returns a list of all objects in the table
.
tabfile_info/1
Returns information about the table dumped to file by tab2file/2 or tab2file/3
The following items are returned:
The name of the dumped table. If the table was a named table, a table with the same name cannot exist when the table is loaded from file with file2tab/2. If the table is not saved as a named table, this field has no significance at all when loading the table from file.
set
, bag
,
duplicate_bag
or ordered_set
). This type will be used
when loading the table again.private
,
protected
or public
). A table loaded from the file
will get the same protection.true
if the table was a named table when dumped
to file, otherwise false
. Note that when a named table
is loaded from a file, there cannot exist a table in the
system with the same name.keypos
of the table dumped to file, which
will be used when loading the table again.public
table need
not correspond to the number of objects actually saved to the
file, as objects might have been added or deleted by another
process during table dump.object_count
and
md5sum
.{Major ,Minor }
containing the major and
minor version of the file format for ets table dumps. This
version field was added beginning with stdlib-1.5.1, files
dumped with older versions will return {0,0}
in this
field.An error is returned if the file is inaccessible, badly damaged or not an file produced with tab2file/2 or tab2file/3.
table/1
table/2
Returns a QLC (Query List
Comprehension) query handle. The module qlc
implements
a query language aimed mainly at Mnesia but ETS tables, Dets
tables, and lists are also recognized by QLC as sources of
data. Calling ets:table/1,2
is the means to make the
ETS table Tab
usable to QLC.
When there are only simple restrictions on the key position
QLC uses ets:lookup/2
to look up the keys, but when
that is not possible the whole table is traversed. The
option traverse
determines how this is done:
-
first_next
. The table is traversed one key at a time by callingets:first/1
andets:next/2
. -
last_prev
. The table is traversed one key at a time by callingets:last/1
andets:prev/2
. -
select
. The table is traversed by callingets:select/3
andets:select/1
. The optionn_objects
determines the number of objects returned (the third argument ofselect/3
); the default is to return100
objects at a time. The match_spec (the second argument ofselect/3
) is assembled by QLC: simple filters are translated into equivalent match_specs while more complicated filters have to be applied to all objects returned byselect/3
given a match_spec that matches all objects. -
{select,
. As forMatchSpec }select
the table is traversed by callingets:select/3
andets:select/1
. The difference is that the match_spec is explicitly given. This is how to state match_specs that cannot easily be expressed within the syntax provided by QLC.
The following example uses an explicit match_spec to traverse the table:
9>true = ets:insert(Tab = ets:new(t, []), [{1,a},{2,b},{3,c},{4,d}]),
MS = ets:fun2ms(fun({X,Y}) when (X > 1) or (X < 5) -> {Y} end),
QH1 = ets:table(Tab, [{traverse, {select, MS}}]).
An example with implicit match_spec:
10> QH2 = qlc:q([{Y} || {X,Y} <- ets:table(Tab), (X > 1) or (X < 5)]).
The latter example is in fact equivalent to the former which
can be verified using the function qlc:info/1
:
11> qlc:info(QH1) =:= qlc:info(QH2).
true
qlc:info/1
returns information about a query handle,
and in this case identical information is returned for the
two query handles.
test_ms/2
This function is a utility to test a
match_spec used in
calls to ets:select/2
. The function both tests
for "syntactic" correctness and runs the
match_spec against the object
. If the match_spec
contains errors, the tuple {error,
is returned
where
is a list of natural language
descriptions of what was wrong with the match_spec. If the
match_spec is syntactically OK, the function returns
{ok,
where
is what would have been
the result in a real ets:select/2
call or false
if the match_spec does not match the object
.
This is a useful debugging and test tool, especially when
writing complicated ets:select/2
calls.
to_dets/2
Fills an already created/opened Dets table with the objects
in the already opened ETS table named
. The Dets
table is emptied before the objects are inserted.
This function provides an efficient way to update one or more counters, without the hassle of having to look up an object, update the object by incrementing an element and insert the resulting object into the table again. (The update is done atomically; i.e. no process can access the ets table in the middle of the operation.)
It will destructively update the object with key
in the table
by adding
to the element
at the
:th position. The new counter value is
returned. If no position is specified, the element directly
following the key (<keypos>+1
) is updated.
If a
is specified, the counter will be
reset to the value
if the following
conditions occur:
- The
is not negative (Incr >= 0
) and the result would be greater than (>
)Threshold - The
is negative (Incr < 0
) and the result would be less than (<
)Threshold
A list of
can be supplied to do several update
operations within the object. The operations are carried out in the
order specified in the list. If the same counter position occurs
more than one time in the list, the corresponding counter will thus
be updated several times, each time based on the previous result.
The return value is a list of the new counter values from each
update operation in the same order as in the operation list. If an
empty list is specified, nothing is updated and an empty list is
returned. If the function should fail, no updates will be done at
all.
The given
is used to identify the object by either
matching the key of an object in a set
table,
or compare equal to the key of an object in an
ordered_set
table (see
lookup/2 and
new/2
for details on the difference).
The function will fail with reason badarg
if:
- the table is not of type
set
orordered_set
, - no object with the right key exists,
- the object has the wrong arity,
- the element to update is not an integer,
- the element to update is also the key, or,
- any of
,Pos
,Incr
orThreshold
is not an integerSetValue
This function provides an efficient way to update one or more elements within an object, without the hassle of having to look up, update and write back the entire object.
It will destructively update the object with key
in the table
. The element at the
:th position
will be given the value
.
A list of {
can be supplied to update several
elements within the same object. If the same position occurs more
than one in the list, the last value in the list will be written. If
the list is empty or the function fails, no updates will be done at
all. The function is also atomic in the sense that other processes
can never see any intermediate results.
The function returns true
if an object with the key
was found, false
otherwise.
The given
is used to identify the object by either
matching the key of an object in a set
table,
or compare equal to the key of an object in an
ordered_set
table (see
lookup/2 and
new/2
for details on the difference).
The function will fail with reason badarg
if:
- the table is not of type
set
orordered_set
,
is less than 1 or greater than the object arity, or,Pos - the element to update is also the key
- all
- delete
- delete-1
- delete_all_objects
- delete_object
- file2tab
- file2tab-1
- first
- foldl
- foldr
- from_dets
- fun2ms
- give_away
- i
- i-1
- info
- info-1
- init_table
- insert
- insert_new
- is_compiled_ms
- last
- lookup
- lookup_element
- match
- match-1
- match-2
- match_delete
- match_object
- match_object-1
- match_object-2
- match_spec_compile
- match_spec_run
- member
- new
- next
- prev
- rename
- repair_continuation
- safe_fixtable
- select
- select-1
- select-2
- select_count
- select_delete
- select_reverse
- select_reverse-1
- select_reverse-2
- setopts
- slot
- tab2file
- tab2file-1
- tab2list
- tabfile_info
- table
- table-1
- test_ms
- to_dets