Rohit Ravi

Neural-symbolic reasoning prototypes

Exploring cellular automata, attention representations, rule mining, and symbolic solvers as components of reasoning systems. Benchmark evaluation remains in progress.

In progress
Neural-Symbolic
Reasoning
LLMs
Research
Sat Jun 01 2024 00:00:00 GMT+0000 (Coordinated Universal Time)

At a glance

Question
Can learned representations support explicit, inspectable symbolic reasoning?
My contribution
Explored cellular automata, attention representations, rule mining, and symbolic solvers.
Result & scope
Prototype work. Benchmark evaluation remains in progress.
In this write-up

Exploratory architectures for reasoning systems that sit between pure neural methods and pure symbolic methods — built around the hypothesis that the right intermediate representation for LLM reasoning is a rule-mined causal graph extracted from attention, not free-form chain-of-thought.

Module 1 — ASI (Recursive CA + Attention + Symbolic)

Phased architecture:

  1. 2D cellular automaton as a controllable dynamical substrate.
  2. Transformer trained on the CA trajectories with attention hooks exposed.
  3. Attention tensors → binary causal matrix → temporal causal graph → FP-growth rule mining over that graph.

The hypothesis is that graph representations derived from attention may support rule discovery. Attention weights alone do not establish causal relationships.

Module 2 — ESDE (Enhanced Symbolic Discovery Engine)

Targets AIME / SAT / GSM8K via a neural-symbolic hybrid: Phi-3 / Mistral via Ollama → SketchNegotiator → RuleEngine backed by SymPy / Z3 → Contrastive Evolver → CodeBERT embeddings + FAISS MemoryGraph + FP-growth → FeedbackLoop.

Status

These are exploratory prototypes. Benchmark results, controlled ablations, and end-to-end reliability have not yet been established. The next step is a controlled comparison of the cellular-automata pipeline against a transformer-only baseline.