ARM Support

[mariusmuntean]

Hi and thanks for this image! I found myself needing Redis+RediSearch+Redis JSON+Gears and this image was a godsent.

My only issue is that on my local machine, an M1 Macbook Pro, RedisMod crashes pretty reproducibly, whereas on an Intel Macbook Pro it runs as expected.
I think it has to do with the fact that there's no native ARM build.

I'm running it like this

docker run \
  -p 6379:6379 \
  -v `pwd`:/data \
  -v `pwd`/docker-redis.conf:/usr/local/etc/redis/redis.conf \
  --platform linux/amd64 \
  redislabs/redismod \
  /usr/local/etc/redis/redis.conf

Notice the --platform linux/amd64 \ which I have to specify, otherwise I get

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
docker: Error response from daemon: platform linux/arm64/v8 not supported.

The docker-redis.conf looks like this

# Persistence defaults
save 60 5000

# Load necessary modules
loadmodule /usr/lib/redis/modules/rejson.so
loadmodule /usr/lib/redis/modules/redisearch.so
loadmodule /usr/lib/redis/modules/redisgraph.so
loadmodule /usr/lib/redis/modules/redisgears.so Plugin /var/opt/redislabs/modules/rg/plugin/gears_python.so


# Generated by CONFIG REWRITE
protected-mode no
user default on nopass ~* &* +@all

In my application I'm using Redis Gears to add a new property to all my Json documents in Redis (a few hundred documents) and then I'm updating the RedisSearch index to include that field.

It always fails with this error on my M1 Macbook Pro:

=== REDIS BUG REPORT START: Cut & paste starting from here ===
1:M 08 Jan 2023 15:48:00.577 # Redis 6.2.6 crashed by signal: 11, si_code: 1
1:M 08 Jan 2023 15:48:00.577 # Accessing address: 0x620008
1:M 08 Jan 2023 15:48:00.577 # Crashed running the instruction at: 0x4005eae9c4

------ STACK TRACE ------
EIP:
/usr/lib/redis/modules/redisgraph.so(DocTable_Free+0x34)[0x4005eae9c4]

Backtrace:
/lib/x86_64-linux-gnu/libpthread.so.0(+0x14140)[0x4002158140]
/usr/lib/redis/modules/redisgraph.so(DocTable_Free+0x34)[0x4005eae9c4]
/usr/lib/redis/modules/redisgraph.so(IndexSpec_FreeInternals+0x9b)[0x4005e52c8b]
/usr/lib/redis/modules/redisgraph.so(RediSearch_DropIndex+0x13)[0x4005e4dc13]
/usr/lib/redis/modules/redisgraph.so(Index_Construct+0x1f)[0x4004d5c8cf]
/usr/lib/redis/modules/redisgraph.so(+0x1f507f)[0x4004d2407f]
/usr/lib/redis/modules/redisgraph.so(+0x245da4)[0x4004d74da4]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8ea7)[0x400214cea7]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x4002263def]

------ REGISTERS ------
1:M 08 Jan 2023 15:48:00.581 # 
RAX:0000000000002780 RBX:0000000000620000
RCX:0000000000000001 RDX:0000000000000003
RDI:000000000061ffc8 RSI:0000004000000000
RBP:0000000000000278 RSP:000000400a4b5430
R8 :0000000000000128 R9 :0000004002600900
R10:000000400a4b5ec8 R11:0000000000000001
R12:000000403628b208 R13:000000405109c000
R14:000000400e1bcaa0 R15:0000004036281000
RIP:0000004005eae9c4 EFL:0000000000000206
CSGSFS:002b000000000033
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543f) -> 0000000000000000
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543e) -> 000000401a827310
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543d) -> 0000004051092038
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543c) -> 000000405109c000
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543b) -> 0000004004d5c8cf
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543a) -> 0000004051092038
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5439) -> 0000004005e4dc13
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5438) -> 000000400e1bcaa0
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5437) -> 000000405109c000
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5436) -> 000000403f295038
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5435) -> 000000401a827310
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5434) -> 000000403628b180
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5433) -> 0000004005e52c8b
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5432) -> 000000403f295038
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5431) -> 000000403628b180
1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5430) -> 000000400646dbc0

------ INFO OUTPUT ------
# Server
redis_version:6.2.6
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:9c335ca9779faba5
redis_mode:standalone
os:Linux 5.15.49-linuxkit x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:10.2.1
process_id:1
process_supervised:no
run_id:44be8be77c77ec3c262115a20f0a977abade217c
tcp_port:6379
server_time_usec:1673192880574820
uptime_in_seconds:16
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:12248496
executable:/data/redis-server
config_file:/usr/local/etc/redis/redis.conf
io_threads_active:0

# Clients
connected_clients:10
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:455
client_recent_max_output_buffer:0
blocked_clients:1
tracking_clients:0
clients_in_timeout_table:0

# Memory
used_memory:16755232
used_memory_human:15.98M
used_memory_rss:0
used_memory_rss_human:0B
used_memory_peak:16831848
used_memory_peak_human:16.05M
used_memory_peak_perc:99.54%
used_memory_overhead:9480206
used_memory_startup:9243472
used_memory_dataset:7275026
used_memory_dataset_perc:96.85%
allocator_allocated:17660248
allocator_active:19320832
allocator_resident:22544384
total_system_memory:4124385280
total_system_memory_human:3.84G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.09
allocator_frag_bytes:1660584
allocator_rss_ratio:1.17
allocator_rss_bytes:3223552
rss_overhead_ratio:0.00
rss_overhead_bytes:-22544384
mem_fragmentation_ratio:0.00
mem_fragmentation_bytes:-16619856
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:205110
mem_aof_buffer:0
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:585
rdb_bgsave_in_progress:0
rdb_last_save_time:1673192864
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:10
total_commands_processed:1235
instantaneous_ops_per_sec:1
total_net_input_bytes:3726
total_net_output_bytes:13122
instantaneous_input_kbps:0.11
instantaneous_output_kbps:0.07
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:1
evicted_keys:0
keyspace_hits:1761
keyspace_misses:5
pubsub_channels:1
pubsub_patterns:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:10
dump_payload_sanitizations:0
total_reads_processed:22
total_writes_processed:25
io_threaded_reads_processed:0
io_threaded_writes_processed:0

# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:6c1515e51f11e042170112e9ec841acaf04cae49
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:0.214587
used_cpu_user:1.225623
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.008020
used_cpu_user_main_thread:0.178458

# Modules
module:name=rg,ver=10204,api=1,filters=1,usedby=[rg],using=[rg],options=[]
module:name=graph,ver=20815,api=1,filters=0,usedby=[],using=[ReJSON],options=[]
module:name=search,ver=999999,api=1,filters=0,usedby=[],using=[ReJSON],options=[handle-io-errors]
module:name=ReJSON,ver=20011,api=1,filters=0,usedby=[search|graph],using=[],options=[handle-io-errors]

# Commandstats
cmdstat_rg.pyexecute:calls=1,usec=256707,usec_per_call=256707.00,rejected_calls=0,failed_calls=0
cmdstat_graph.QUERY:calls=5,usec=206738,usec_per_call=41347.60,rejected_calls=0,failed_calls=0
cmdstat_select:calls=4,usec=222,usec_per_call=55.50,rejected_calls=0,failed_calls=0
cmdstat_info:calls=17,usec=3251,usec_per_call=191.24,rejected_calls=0,failed_calls=0
cmdstat_config:calls=11,usec=2438,usec_per_call=221.64,rejected_calls=0,failed_calls=0
cmdstat_cluster:calls=5,usec=180,usec_per_call=36.00,rejected_calls=0,failed_calls=5
cmdstat_echo:calls=10,usec=248,usec_per_call=24.80,rejected_calls=0,failed_calls=0
cmdstat_get:calls=5,usec=447,usec_per_call=89.40,rejected_calls=0,failed_calls=0
cmdstat_json.get:calls=581,usec=16688,usec_per_call=28.72,rejected_calls=0,failed_calls=0
cmdstat_json.set:calls=581,usec=31328,usec_per_call=53.92,rejected_calls=0,failed_calls=0
cmdstat_subscribe:calls=5,usec=574,usec_per_call=114.80,rejected_calls=0,failed_calls=0
cmdstat_client:calls=10,usec=434,usec_per_call=43.40,rejected_calls=0,failed_calls=0

# Errorstats
errorstat_ERR:count=10

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=585,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------
id=15 addr=172.17.0.1:60934 laddr=172.17.0.2:6379 fd=16 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=7 idle=0 flags=b db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=69 obl=0 oll=0 omem=0 tot-mem=61541 events=r cmd=graph.QUERY user=default redir=-1
id=16 addr=172.17.0.1:60950 laddr=172.17.0.2:6379 fd=17 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=7 idle=6 flags=P db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=subscribe user=default redir=-1
id=17 addr=172.17.0.1:60966 laddr=172.17.0.2:6379 fd=18 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=info user=default redir=-1
id=18 addr=172.17.0.1:60970 laddr=172.17.0.2:6379 fd=19 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=P db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=subscribe user=default redir=-1
id=19 addr=172.17.0.1:60986 laddr=172.17.0.2:6379 fd=20 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=info user=default redir=-1
id=20 addr=172.17.0.1:61000 laddr=172.17.0.2:6379 fd=21 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=P db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=subscribe user=default redir=-1
id=21 addr=172.17.0.1:61014 laddr=172.17.0.2:6379 fd=22 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=info user=default redir=-1
id=22 addr=172.17.0.1:61022 laddr=172.17.0.2:6379 fd=23 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=P db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=subscribe user=default redir=-1
id=23 addr=172.17.0.1:61034 laddr=172.17.0.2:6379 fd=24 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=3 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=rg.pyexecute user=default redir=-1
id=24 addr=172.17.0.1:61036 laddr=172.17.0.2:6379 fd=25 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=P db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=subscribe user=default redir=-1

------ MODULES INFO OUTPUT ------
# rg
rg_nexecutions:0
rg_nregistrations:0

# rg_regisrations

# rg_plugins
rg_GearsPythonPlugin:version=10000

# rg_python_stats
rg_TotalAllocated:31961672
rg_PeakAllocated:8576320
rg_CurrAllocated:8515464

# rg_python_requirements

# rg_python_sessions

# graph_executing commands
graph_command:GRAPH.QUERY CREATE INDEX ON :mentioned(RelationshipType)

# ReJSON_trace
ReJSON_trace:   0: redis_module::base_info_func
   1: rejson::__info_func
   2: modulesCollectInfo
             at /usr/src/redis/src/module.c:7078:9
   3: logModulesInfo
             at /usr/src/redis/src/debug.c:1598:22
   4: printCrashReport
             at /usr/src/redis/src/debug.c:1849:5
      sigsegvHandler
             at /usr/src/redis/src/debug.c:1831:5
   5: <unknown>
   6: DocTable_Free
   7: IndexSpec_FreeInternals
   8: RediSearch_DropIndex
   9: Index_Construct
  10: _ExecuteQuery
  11: thread_do
  12: start_thread
  13: clone


------ FAST MEMORY TEST ------
1:M 08 Jan 2023 15:48:00.636 # main thread terminated
1:M 08 Jan 2023 15:48:00.637 # Bio thread for job type #0 terminated
1:M 08 Jan 2023 15:48:00.637 # Bio thread for job type #1 terminated
1:M 08 Jan 2023 15:48:00.637 # Bio thread for job type #2 terminated

Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

------ DUMPING CODE AROUND EIP ------
Symbol: DocTable_Free (base: 0x4005eae990)
Module: /usr/lib/redis/modules/redisgraph.so (base 0x4004b2f000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x4005eae990 -D -b binary -m i386:x86-64 /tmp/dump.bin
------
1:M 08 Jan 2023 15:48:00.639 # dump of function (hexdump of 180 bytes):
41544989fc5549837c24180053488b7f30743f31ed0f1f004889e848c1e004488b1c074885db741f0f1f840000000000488d7bc8488b5b08e8c3edffff4885db75ee498b7c24304883c50149396c241877c6488b0517d95600ff105b498d7c24385d415ce987ffffff0f1f8000000000488b3f488d0db6e9ffff0fb7d2e9fe21feff0f1f4000662e0f1f840000000000415641554989d641544c8d673855534889fd4989f54c89e7e833f5ffff4885c00f841a01
Function at 0x4005ead790 is DMD_Free
Function at 0x4005eadf70 is DocIdMap_Get

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

[epinzur]

I'm seeing similar issues using RedisAI on apple silicon. It would be great to get ARM support for RedisAI.

[DjellalAbdou]

am having the same issue, is there a workaround for it until ARM support is added?