Fields¶
The core module of limpyd
provides 6 fields types, matching the ones in Redis:
- StringField, for the main data type in Redis, strings
- HashField, for dicts
- InstanceHashField, for hashes
- SetField, for sets
- ListField, for lists
- SortedSetField, for sorted sets
You can also manage primary keys with these too fields:
- PKField, based on StringField
- AutoPKField, same as PKField but auto-incremented.
All these fields can be indexed, and they manage the keys for you (they take the same arguments as the real Redis ones, as defined in the StrictRedis
class of redis-py, but without the key
parameter).
Another thing all fields have in common, is the way to delete them: use the delete
method on a field, and both the field and its value will be removed from Redis.
Field attributes¶
When adding fields to a model, you can configure it with some attributes:
default¶
It’s possible to set default values for fields of type StringField and InstanceHashField:
class Example(model.RedisModel):
database = main_database
foo = fields.StringField(default='FOO')
bar = fields.StringField()
>>> example = Example(bar='BAR')
>>> example.foo.get()
'FOO'
When setting a default value, the field will be saved when creating the instance. If you defined a PKField (not AutoPKField), don’t forget to pass a value for it when creating the instance, it’s needed to store other fields.
indexable¶
Sometimes getting objects from Redis by its primary key is not what you want. You may want to search for objects with a specific value for a specific field.
By setting the indexable
argument to True
when defining the field, this feature is automatically activated, and you’ll be able to retrieve objects by filtering on this field using Collections.
To activate it, just set the indexable
argument to True
:
class Example(model.RedisModel):
database = main_database
foo = fields.StringField(indexable=True)
bar = fields.StringField()
In this example you will be able to filter on the field foo
but not on bar
.
When updating an indexable field, a lock is acquired on Redis on this field, for all instances of the model. It isn’t possible for this to use pipeline or redis scripting, because both need to know in advance the keys to update, but we don’t always know since keys for indexes may be based on values. So all writing operations on an indexable field are protected, to ensure consistency if many threads, process, servers are working on the same Redis database.
If you are sure you have only one thread, or you don’t want to ensure consistency, you can disable locking by setting to False
the lockable
argument when creating a field, or the lockable
attribute of a model to inactive the lock for all of its fields.
unique¶
The unique
argument is the same as the indexable
one, except it will ensure that you can’t have multiple objects with the same value for some fields. unique
fields are also indexed, and can be filtered, as for the indexable
argument.
Example:
class Example(model.RedisModel):
database = main_database
foo = fields.StringField(indexable=True)
bar = fields.StringField(unique=True)
>>> example1 = Example(foo='FOO', bar='BAR')
True
>>> example2 = Example(foo='FOO', bar='BAR')
UniquenessError: Key :example:bar:BAR already exists (for instance 1)
See Collections to know how to filter objects, as for indexable
.
indexes¶
This allow to change the default index used, or use many of them. See the “Indexing” section in Collections to know more.
lockable¶
You can set this argument to False
if you don’t want a lock to be acquired on this field for all instances of the model. See indexable
for more information about locking.
If not specified, it’s default to True
, except if the lockable
attribute of the model is False
, in which case it’s forced to False
for all fields.
Field types¶
StringField¶
StringField based fields allow the storage of strings, but some Redis string commands allow to treat them as integer, float [1] or bits.
Example:
from limpyd import model, fields
class Example(model.RedisModel):
database = main_database
name = fields.StringField()
You can use this model like this:
>>> example = Example(name='foo')
>>> example.name.get()
'foo'
>>> example.name.set('bar')
>>> example.name.get()
'bar'
>> example.name.delete()
True
The StringField type support these Redis string commands:
Getters¶
bitcount
get
getbit
getrange
getset
strlen
HashField¶
HashField allows storage of a dict in Redis.
Example:
class Email(model.RedisModel):
database = main_database
headers = fields.HashField()
>>> email = Email()
>>> headers = {'from': 'foo@bar.com', 'to': 'me@world.org'}
>>> email.headers.hmset(**headers)
>>> email.headers.hget('from')
'foo@bar.com'
The HashField type support these Redis hash commands:
Getters¶
hget
hgetall
hmget
hkeys
hvals
hexists
hlen
hscan
(returns a generator with all/matching key/value pairs, you don’t have to manage the cursor)
InstanceHashField¶
As for StringField, InstanceHashField based fields allow the storage of strings. But all the InstanceHashField fields of an instance are stored in the same Redis hash, the name of the field being the key in the hash.
To fully use the power of Redis hashes, we also provide two methods to get and set multiples field in one operation (see hmget and hmset). It’s usually cheaper to store fields in hash that in strings. And it’s faster to set/retrieve them using these two commands.
Example with simple commands:
class Example(model.RedisModel):
database = main_database
foo = fields.InstanceHashField()
bar = fields.InstanceHashField()
>>> example.foo.hset('FOO')
1 # 1 because the hash field was created
>>> example.foo.hget()
'FOO'
The InstanceHashField type support these Redis hash commands:
Getters¶
hget
Deleter¶
To delete the value of a InstanceHashField, you can use the hdel
command, which do the same as the main `delete`
one.
See also hdel on the model to delete many InstanceHashField at once
Multi¶
The following commands are not called on the fields themselves, but on an instance:
hmget¶
hmget is called directly on an instance, and expects a list of field names to retrieve.
The result will be, as in Redis, a list of all values, in the same order.
If no names are provided, nothing will be fetched. Use hvals, or better, hgetall to get values for all InstanceHashFields
It’s up to you to associate names and values, but you can find an example below:
class Example(model.RedisModel):
database = main_database
foo = fields.InstanceHashField()
bar = fields.InstanceHashField()
baz = fields.InstanceHashField()
qux = fields.InstanceHashField()
def hmget_dict(self, *args):
"""
A call to hmget but which return a dict with field names as keys, instead
of only a list of values
"""
values = self.hmget(*args)
keys = args or self._hashable_fields
return dict(zip(keys, values))
>>> example = Example(foo='FOO', bar='BAR')
>>> example.hmget('foo', 'bar')
['FOO', 'BAR']
>>> example.hmget_dict('foo', 'bar')
{'bar': 'BAR', 'foo': 'FOO'}
hmset¶
hmset is the reverse of hmget, and also called directly on an instance, and expects named arguments with field names as keys, and new values to set as values.
Example (with same model as for hmget):
>>> example = Example()
>>> example.hmset(foo='FOO', bar='BAR')
True
>>> example.hmget('foo', 'bar')
['FOO', 'BAR']
hdel¶
hdel is called directly on an instance, and expects a list of field names to delete.
The result will be, as in Redis, the number of field really deleted (ie fields without any stored value won’t be taken into account).
>>> example = Example()
>>> example.hmset(foo='FOO', bar='BAR', baz='BAZ')
True
>>> example.hmget('foo', 'bar', 'baz')
['FOO', 'BAR', 'BAZ']
>>> example.hdel('foo', 'bar', 'qux')
2
>>> example.hmget('foo', 'bar', 'baz')
[None, None, 'BAZ']
Note that you can also call hdel on an InstanceHashField itself, without parameters, to delete this very field.
>>> example.baz.hdel()
1
hgetall¶
hgetall must be called directly on an instance, and will return a dictionary containing names and values of all InstanceHashField with a stored value.
If a field has no stored value, it will not appear in the result of hgetall.
Example (with same model as for hmget):
>>> example = Example(foo='FOO', bar='BAR')
>>> example.hgetall()
{'foo': 'FOO', 'bar': 'BAR'}
>>> example.foo.hdel()
>>> example.hgetall()
{bar': 'BAR'}
hkeys¶
hkeys must be called on an instance and will return the name of all the InstanceHashField with a stored value.
If a field has no stored value, it will not appear in the result of hkeys.
Note that the result is not ordered in any way.
Example (with same model as for hmget):
>>> example = Example(foo='FOO', bar='BAR')
>>> example.hkeys()
['foo', 'bar']
>>> example.foo.hdel()
>>> example.hkeys()
['bar']
hvals¶
hkeys must be called on an instance and will return the value of all the InstanceHashField with a stored value.
If a field has no stored value, it will not appear in the result of hvals.
Note that the result is not ordered in any way.
Example (with same model as for hmget):
>>> example = Example(foo='FOO', bar='BAR')
>>> example.hvals()
['FOO', 'BAR']
>>> example.foo.hdel()
>>> example.hvals()
['BAR']
hlen¶
hlen must be called on an instance and will return the number of InstanceHashField with a stored value.
If a field has no stored value, it will not be count in the result of hlen.
Example (with same model as for hmget):
>>> example = Example(foo='FOO', bar='BAR')
>>> example.hlen()
2
>>> example.foo.hdel()
>>> example.hlen()
1
SetField¶
SetField based fields can store many values in one field, using the set data type of Redis, an unordered set (with unique values).
Example:
from limpyd import model, fields
class Example(model.RedisModel):
database = main_database
stuff = fields.SetField()
You can use this model like this:
>>> example = Example()
>>> example.stuff.sadd('foo', 'bar')
2 # number of values really added to the set
>>> example.stuff.smembers()
set(['foo', 'bar'])
>>> example.stuff.sismember('bar')
True
>>> example.stuff.srem('bar')
True
>>> example.stuff.smembers()
set(['foo'])
>>> example.stuff.delete()
True
The SetField type support these Redis set commands:
Getters¶
scard
sismember
smembers
srandmember
sscan
(returns a generator with all/matching values, you don’t have to manage the cursor)sort
(with arguments like in redis-py, see redis-py-sort)
Modifiers¶
sadd
spop
srem
ListField¶
ListField based fields can store many values in one field, using the list data type of Redis. Values are ordered, and are not unique (you can push many times the same value).
Example:
from limpyd import model, fields
class Example(model.RedisModel):
database = main_database
stuff = fields.ListField()
You can use this model like this:
>>> example = Example()
>>> example.stuff.rpush('foo', 'bar')
2 # number of values added to the list
>>> example.stuff.lrange(0, -1)
['foo', 'bar']
>>> example.stuff.lindex(1)
'bar'
>>> example.stuff.lrem(1, 'bar')
1 # number of values really removed
>>> example.stuff.lrange(0, -1)
['foo']
>>> example.stuff.delete()
True
The ListField type support these Redis list commands:
Getters¶
lindex
llen
lrange
sort
(with arguments like in redis-py, see redis-py-sort)
Modifiers¶
linsert
lpop
lpush
lpushx
lrem
lset
ltrim
rpop
rpush
rpushx
SortedSetField¶
SortedSetField based fields can store many values, each scored, in one field using the sorted-set data type of Redis. Values are unique (it’s a set), and are ordered by their score.
Example:
from limpyd import model, fields
class Example(model.RedisModel):
database = main_database
stuff = fields.SortedSetField()
You can use this model like this:
>>> example = Example()
>>> example.stuff.zadd(foo=2.5, bar=1.1)
2 # number of values added to the sorted set
>>> example.stuff.zrange(0, -1)
['bar', 'foo']
>>> example.stuff.zrangebyscore(1, 2, withscores=True)
[('bar', 1.1)]
>>> example.stuff.zrem('bar')
1 # number of values really removed
>>> example.stuff.zrangebyscore(1, 2, withscores=True)
[]
>>> example.stuff.delete()
True
The SortedSetField type support these Redis sorted set commands:
Getters¶
zcard
zcount
zrange
zrangebyscore
zrank
zrevrange
zrevrangebyscore
zrevrank
zscore
zscan
(returns a generator with all/matching key/score pairs, you don’t have to manage the cursor)sort
(with arguments like in redis-py, see redis-py-sort)
Modifiers¶
zadd
zincrby
zrem
zremrangebyrank
zremrangebyscore
PKField¶
PKField is a special subclass of StringField that manage primary keys of models. The PK of an object cannot be updated, as it serves to create keys of all its stored fields. It’s this PK that is returned, with others, in Collections.
A PK can contain any sort of string you want: simple integers, float [1], long uuid, names…
If you want a PKField which will be automatically filled, and auto-incremented, see AutoPKField. Otherwise, with standard PKField, you must assign a value to it when creating an instance.
By default, a model has a AutoPKField attached to it, named pk
. But you can redefine the name and type of PKField you want.
Examples:
class Foo(model.RedisModel):
"""
The PK field is ``pk``, and will be auto-incremented.
"""
database = main_database
class Bar(model.RedisModel):
"""
The PK field is ``id``, and will be auto-incremented.
"""
database = main_database
id = fields.AutoPKField()
class Baz(model.RedisModel):
"""
The PK field is ``name``, and won't be auto-incremented, so you must assign it a value when creating an instance.
"""
database = main_database
name = fields.PKField()
Note that whatever name you use for the PKField (or AutoPKField), you can always access it via the name pk
(but also we its real name). It’s easier for abstraction:
class Example(model.RedisModel):
database = main_database
id = fields.AutoPKField()
name = fields.StringField()
>>> example = Example(name='foobar')
>>> example.pk.get()
1
>>> example.id.get()
1
AutoPKField¶
A AutoPKField field is a PKField filled with auto-incremented integers, starting to 1
. Assigning a value to of AutoPKField is forbidden.
It’s a AutoPKField that is attached by default to every model, if no other PKField is defined.
See PKField for more details.
[1] | (1, 2, 3, 4, 5) When working with floats, pass them as strings to avoid precision problems. |