{"id":98796,"date":"2024-11-14T12:00:00","date_gmt":"2024-11-14T12:00:00","guid":{"rendered":"https:\/\/staging.checkmarx.com\/?p=98796"},"modified":"2026-01-05T19:04:47","modified_gmt":"2026-01-05T17:04:47","slug":"free-hugs-what-to-be-wary-of-in-hugging-face-part-1","status":"publish","type":"zero-post","link":"https:\/\/checkmarx.com\/blog\/free-hugs-what-to-be-wary-of-in-hugging-face-part-1\/","title":{"rendered":"\u201cFree Hugs\u201d \u2013 What To Be Wary of in Hugging Face \u2013 Part 1\u00a0"},"content":{"rendered":"

Introduction <\/h3>\n\n\n\n

GenAI has taken the world by storm. To meet the needs for development of LLM\/GenAI technology through open-source, various vendors have risen to meet the need to spread this technology. <\/p>\n\n\n\n

One well-known platform is Hugging Face \u2013 an open-source platform that hosts GenAI models. It is not unlike GitHub in many ways \u2013 it\u2019s used for serving content (such as models, datasets and code), version control, issue tracking, discussions and more. It also allows running GenAI-driven apps in online sandboxes. It\u2019s very comprehensive and at this point a mature platform chock full of GenAI content, from text to media. <\/p>\n\n\n\n

In this series of blog posts, we will explore the various potential risks present in the Hugging Face ecosystem.<\/strong> <\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

Championing logo design Don\u2019ts (sorry not sorry opinions my own)<\/em> <\/p>\n\n\n\n

This is Part 1 of a four-part serie<\/em>s: Part 2<\/a> ; Part 3<\/a> ; Part 4<\/a><\/em><\/p>\n\n\n\n

Hugging Face Toolbox and Its Risks <\/h2>\n\n\n\n

Beyond hosting models and associated code, Hugging Face is a also maintainer of multiple libraries for interfacing with all this goodness \u2013 libraries for uploading, downloading and executing models to the Hugging Face platform. From a security standpoint \u2013 this offers a HUGE attack surface to spread malicious content through. On that vast attack surface a lot has already been said and many things have been tested in the Hugging Face ecosystem, but many legacy vulnerabilities persist, and bad security practices still reign supreme in code and documentation;  these can bring an organization to its knees (while being practiced by major vendors!) and known issues are shrugged off because \u201cthat\u2019s just the way it is\u201d – while new solutions suffer from their own set of problems.. <\/p>\n\n\n\n

ReadMe.md? More Like \u201cTrustMe.md\u201d <\/h3>\n\n\n\n

The crux of all potentially dangerous behavior around marketplaces and repositories is trust<\/strong> – trusting the content\u2019s host, trusting the content\u2019s maintainer and trusting that no one is going to pwn either. This is also why environments that allow obscuring malicious code or ways to execute it are often more precarious for defenders. <\/p>\n\n\n\n

While downloading things from Hugging Face is trivial, actually using<\/em> them is finnicky \u2013 in that there is no one global definitive way to do so and trying to do it any other way than the one recommended by the vendor will likely end in failure. Figuring out how to use a model always boils down to RTFM \u2013 the ReadMe. <\/p>\n\n\n\n

But can ReadMe files be trusted? Like all code, there are good and bad practices – even major vendors fall for that. For example, Apple actively uses dangerous flags when instructing users on loading their models: <\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

trust_remote_code sounds like a very reasonable flag to set to True<\/em> <\/p>\n\n\n\n

There are many<\/em> ways to dangerously introduce code into the process, simply because users are bound to trust what the ReadMe presents to them. They can load malicious code, load malicious models in a manner that is both dangerous and very obscure. <\/p>\n\n\n\n

Configuration-Based Code Execution Vectors <\/h2>\n\n\n\n

Let\u2019s start by examining the above configurations in its natural habitat.<\/p>\n\n\n\n

 Transformers<\/a> is one of the many tools Hugging Face provides users with, and its purpose is to normalize the process of loading models, tokenizers and more with the likes of AutoModel and AutoTokenizer<\/a>. It wraps around many of the aforementioned technologies and mostly does a good job only utilizing secure calls and flags. <\/p>\n\n\n\n

However<\/strong> \u2013 all of that security goes out the window once code execution for custom models that load as Python code behind a flag, \u201ctrust_remote_code=True\u201d, which allows loading classes for models and tokenizers which require additional code and a custom implementation to run. <\/p>\n\n\n\n

While it sounds like a terrible practice that should be rarely used, this flag is commonly set to True. Apple was already mentioned, so here\u2019s a Microsoft example: <\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

why wouldn’t you trust remote code from Microsoft? What are they going to do, force install Window 11 on y- uh oh it\u2019s installing Windows 11<\/em> <\/p>\n\n\n\n

Using these configurations with an unsecure model could lead to unfortunate results. <\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

Code loads dangerous config <\/em>\u00e0<\/em> config loads code module <\/em>\u00e0<\/em> code loads OS command<\/em> <\/p>\n\n\n\n