Skip to main content

DiceDB - Overview

Purpose:​

DiceDB is an open-source, in-memory database designed for speed, reactivity, and efficient resource utilization. Its primary goal is to provide a high-performance caching solution that can also deliver real-time data updates to clients through query subscriptions, optimized for modern hardware.

Architecture:​

DiceDB's architecture includes the following key components:

  1. Client Connection Handlers: These components manage client connections using protocols like TCP-RESP, HTTP and WebSockets.
  2. Command Processing: Commands received from clients are parsed and validated before execution.
  3. Data Store (Shards): The in-memory data is partitioned across multiple shards (SwissTable or Go maps) for concurrency and scalability.
  4. Query Manager: Oversees and manages query subscriptions and their execution.
  5. Reactive Engine: Detects data changes and triggers updates to subscribed clients (using .WATCH commands and server-sent events or WebSockets).
  6. Write Ahead Log (WAL): Provides data durability by logging write operations, although a null WAL is currently implemented by default.
  7. Lua Scripting: Executes LUA Scripts

Key Functionalities:​

  • Key-Value Storage: Basic operations like SET, GET, DEL, EXISTS, TYPE.
  • String Operations: String manipulation functionalities such as APPEND, GETRANGE, SETBIT, GETBIT, BITCOUNT, BITPOS.
  • List Operations: Functionalities like LPUSH, RPUSH, LPOP, RPOP, LLEN, LINSERT, LRANGE.
  • Hash Operations: Functionalities like HSET, HGET, HGETALL, HDEL, HKEYS, HSTRLEN, HVALS, HEXISTS, HINCRBY, HINCRBYFLOAT.
  • Set Operations: Operations to manage sets using SADD, SREM, SMEMBERS, SCARD, SINTER, SDIFF.
  • Sorted Set Operations: Support for managing sorted sets through commands like ZADD, ZCARD, ZPOPMIN, ZPOPMAX, ZRANGE, ZRANK.
  • JSON Support: Store, retrieve, and manipulate JSON documents natively using commands such as JSON.SET, JSON.GET, JSON.DEL, JSON.MGET, JSON.MSET, JSON.TYPE, JSON.CLEAR, JSON.ARRLEN, JSON.ARRAPPEND, JSON.ARRTRIM, JSON.ARRPOP, JSON.ARRINDEX, JSON.NUMINCRBY, JSON.STRLEN, JSON.OBJKEYS, JSON.OBJLEN, JSON.TOGGLE, JSON.RESP.
  • Geospatial Indexing: Commands like GEOADD and GEODIST for geospatial calculations.
  • Bloom Filters: Support for probabilistic data structures with commands like BF.ADD, BF.EXISTS, BF.RESERVE.
  • Reactivity (Q.WATCH, .WATCH): Support for subscribing to query results and receiving real-time updates when the underlying data changes.
  • Transactions: Atomic execution of multiple operations.
  • Authentication: Access control using the AUTH command.
  • Introspection: COMMAND family for inspecting server commands and their specifications.
  • Utility: ECHO, SLEEP, and DEBUG commands for testing and diagnostics.
  • Key Management: Expiry and persistence functionalities with EXPIRE, EXPIREAT, TTL, PTTL, and PERSIST commands.
  • Scaling and Concurrency: Multi-threading support and sharding.