Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Deepity 0.1.0a3 releases predictive coding engine

AI summary

A solo developer released Deepity 0.1.0a3, an open-source predictive coding library with a C++20 core, hand-written CPU SIMD kernels and Python bindings published on PyPI as pydeepity. In the author's matched MNIST test, Deepity reached 98.21% accuracy in 487.5 seconds, compared with 97.55% for jpc and 98% for ngc-learn. The author ran the tests on a 12th-generation Intel Core i7-12700H laptop CPU and made the logs reproducible from the repository. The alpha CUDA backend builds and runs, but large-scale ImageNet training remains unavailable while it is being optimized, and no CUDA-versus-CPU results were published.

In-depth insight

The release makes an open-source predictive coding implementation with kernel-level CPU optimization publicly available for inspection and experimentation.

I've been building a predictive coding (PC) engine from scratch in C++20, with a thin Python layer on top, and just cut the first public release. Posting here for feedback and because I haven't seen many from-scratch PC implementations with real kernel-level optimization work behind them.

What predictive coding is: instead of backprop's single global error signal flowing backward through a stored computation graph, each layer in a PCN holds a local belief about its own state and compares it against a top-down prediction from the layer above. Training alternates between "settling" (every layer locally reduces its own prediction error over several steps) and a weight update that only looks at that layer's own error. No layer needs the transpose of another layer's weights, and there's no backward graph to keep in memory.

What Deepity actually is:

- A C++20 core with hand-written AVX2/AVX-512 SIMD kernels (via SLEEF for the transcendental activations), OpenBLAS/MKL for GEMM, and a CUDA backend (cuBLAS + CUTLASS + CUDA graph capture for the settling loop) that's still being tuned for large-scale training (not available yet).

#Deepity 0.1.0a3#pydeepity#jpc#ngc-learn#PyTorch#CUDA#OpenBLAS#Intel MKL#SLEEF#nanobind

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.