printLLMs plus Coding Agents = Security Nightmare
by kleinbl00
In 2023, security researcher Bar Lanyado noted that LLMs hallucinated a package named "huggingface-cli". While this name is identical to the command used for the command-line version of HuggingFace Hub, it is not the name of the package. The software is correctly installed with the code pip install -U "huggingface_hub[cli]". Lanyado tested the potential for slopsquatting by uploading an empty package under this hallucinated name. In three months, it had received over 30,000 downloads. The hallucinated packaged name was also used in the README file of a repo for research conducted by Alibaba.