Databricks injects domain knowledge into prompts to improve LLM npm malware triage
A Databricks blog post describes findings and fixes from the LLM triage stage of its npm malware scanning pipeline. Collecting samples since November 2025, the team found that of 11,413 package-versions corroborated through OSV npm malware advisories, 96.2% hide their payload in the package source or fetch it from an external server, while only 3.8% (432 package-versions) bundle a binary. Those binary-bearing packages are the hardest to judge: among OSV-confirmed samples they failed to reach a decisive malicious verdict 14.4% of the time, more than double the 6.2% rate for source-only packages. The team built five adversarial Node.js native addon (.node) payload types covering credential theft, dropper/backdoor, reverse shell, cryptominer stub and persistence, with an obfuscation ladder running from a readable baseline to stripped symbols and single-byte XOR-encoded strings, in PE, ELF (all stripped) and Mach-O formats. Rather than fine-tuning models, they inject pre-computed static evidence into the prompt before inference, which they say improves detection quality for both budget and frontier models.
It quantifies a specific LLM failure mode in supply-chain triage and points to a low-cost prompt-level fix that avoids fine-tuning.