Analysis

Chinese AI companies are barraging US companies with distillation attacks to extract model weights and other sensitive information. Yet while distillation attacks may be new, the logic behind them is not. Distillation attacks exploit traditional IP theft techniques against a new technology target. As with myriad other industries throughout history, companies that can steal IP and thereby minimize their R&D costs gain an immediate advantage over competitors that must do their own R&D.. 

This is particularly the case when the theft is challenging to ascertain. With distillation attacks, for example, individual queries of AI models are legal. The challenge is that distillation occurs at mass scale using AI agents that disguise their intentions. Such attacks are often undetected but pose a massive financial and compliance risk to AI companies whose models may inadvertently leak sensitive information. The incentive is therefore for AI companies to engage in large-scale distillation, particularly in the PRC where weak rule of law limits the consequences of such attacks. 

Some companies realize that distillation might not be worth the potential consequences. ByteDance’s founder, for example, explicitly discouraged distillation in a recent interview with Chinese state media, arguing that the short-term gains from distillation do not outweigh the longer-term benefits of developing the company’s own models. In part, this is an attempt to publicly distance the company from accusations of IP theft that could result in US sanctions. But it is also a signal to a domestic audience in the PRC. Smaller models might be outpacing ByteDance now thanks to distillation, but ByteDance intends to be in this race for the long haul. From a political perspective, ByteDance’s founder Zhang Yiming is also signalling to CCP leaders that the company intends to build a truly indigenous model, which will appeal to some party elites and help to insulate the company against political accusations that it is lagging in the intense race to outpace US innovation. 

The challenge for ByteDance is that such logic will likely face an unfavorable reception if the company cannot demonstrate that it is continuing to outpace its competitors in both the United States and in the PRC. Smaller PRC models are striving for nimbleness and broad usage at the expense of tech giants, and often with smaller development or operating costs because they are also employing extensive distillation attacks. 

AI development in the PRC is therefore currently a Wild West-style gold rush in which the immense rewards from minimizing R&D costs incentivize distillation attacks seemingly without recourse. If widespread distillation attacks mean that research costs are increasingly unlikely to be recuperated as some companies leapfrog expensive R&D processes through IP theft, then intensified and unfair competition from Chinese companies will escalate. 

This means that structural issues in the Chinese economy will continue to underscore the AI sector. Overcapacity and a hyper competitive industry incentivize a “race to the bottom” on profit margins at home while relying on selling abroad to attempt to offset lost domestic revenue. This is prevalent in industries from automotives to solar panels. China’s AI landscape appears to be heading in a similar direction. 

The next step is therefore that Chinese AI companies will try to flood foreign markets with cheap alternatives built on stolen Western IP. In other words, a new “China Shock” as previously experienced by manufacturing and other related industries is emerging. AI companies in both the US and the PRC will need to learn how to defend themselves fast to prevent not only the ongoing, parasitic theft of their technologies, but also being put out of business by a boom in competitors offering similar products at a fraction of the cost.

Analysis

The US government is trying to force allies and partners to choose a side and exclusively use US or Chinese AI models. While the reasoning is to prevent distillation or other leaking of potentially sensitive information, or what some have termed a protection of “digital sovereignty,” the pressure on US partners to exclude PRC models will likely ruffle feathers. 

First, US allies and other partners are currently broadly unhappy with the United States’ foreign policy, ranging from ire against tariffs to questioning the United States’ commitment to democracy and longstanding alliances. This means that pressure to see AI as zero sum (in other words, “us or them”) will likely be met with suspicion that the US is continuing to treat its allies like minions rather than partners. 

The White House’s seemingly naked power play in demanding adherence to US models will likely find a similar reception to the imposition of tariffs or other policies; partners will be upset. 

This has several implications both for how the United States conducts foreign policy and for how US businesses operate in global markets. First, countries facing this choice may well choose Chinese models over their US counterparts. That would then push a country towards the PRC and potentially exclude US companies from operating fully in those territories. 

Second, coercion to choose US models may incentivize other countries to support the development of their own domestic models as a function of national security. If a NATO ally, for example, sees US coercion as threatening, particularly if US models are used in defense or other sensitive sectors, then the incentive is to create an alternative domestic model that excludes US-origin technology. In short, by pushing for exclusivity in how its partners use AI models, rather than encouraging use through the strength and usability of superior US models, may prove counterproductive. 

This should be of particular concern for US companies that could be viewed as extensions of the US government in how they are treated by foreign partners. This will make for a messier regulatory and business environment in which a company’s national origin factors into who wins contracts or opportunities. 

This already happens with Chinese companies, in which potential ties to the Chinese government rightly color the kinds of engagements that other companies and governments around the world are willing to engage with. The risk of US companies being treated in a similar manner is growing.

Analysis

The latest numbers on the Chinese economy suggest a striking split between domestic and foreign markets. Between January and July of this year, China’s year-on-year indicators for domestic economic strength nearly uniformly suffered: automobile sales dropped 24 percent, fixed-asset investment decreased 6.7 percent, and property investment dropped 19.2 percent; meanwhile, unemployment ticked up 0.2 points to 5.2 percent. This is a stark contrast to China’s July export numbers, which marked a nearly 25 percent increase, largely thanks to its booming AI industry. 

Although Chinese government statistics merit caution because of systematic distortions in collection as well politicization, these figures paint a concerning picture even as approximate measures of the Chinese economy. 

Deflationary pressures in the Chinese domestic market are both discouraging domestic demand and moving products into international markets with depressed prices. The result is a massive spike in exports and the disruption of international markets by unfairly priced Chinese goods. China may cut interest rates by a quarter point to increase inflation (or reduce deflation) but that is a weak effort, and no broader stimulus package is currently expected. This means that domestic deflation and low prices for exports will persist. US companies need to be prepared to litigate or compete with the artificially deflated prices of Chinese goods. 

Another alarming indicator of Chinese domestic economic uncertainty is that none of the world’s ten largest private-equity firms disclosed any investments into China during the first half of this year. Firms are rightly spooked by China’s recent unwinding of Meta’s attempted acquisition of the Chinese firm Manus, as well as the increased regulatory and reputational risks global firms now face when doing business with or in China. 

While new private capital investments seek new markets, it is becoming increasingly difficult to withdraw existing assets from China. Following the Manus debacle, for example, China passed new laws preventing foreign companies from relocating their assets if the Chinese government believes a company is doing so in response to political pressure from their home governments. How this is determined remains opaque and, like all other Chinese laws, is open to broad interpretation to fit government needs. The result is that as the risk of doing business with China increases, avenues for derisking or decoupling are contracting.

Analysis

China’s love affair with open weight models may be coming to an end. Recent instances as with OpenAI and Hugging Face, in which an LLM escaped a sandbox environment, appear to have spooked the CCP (and the US government) into taking AI regulation more seriously. Chinese open weight models may therefore face increasing restrictions at home out of concerns for data security, consumer protections, and the ability for the CCP to maintain political control over model outputs. Despite the CCP’s vocal championing of open weight models, it is not clear that this approach will survive the party’s paranoid necessity to ensure its own political control. 

The CCP is also signaling its intent to restrict foreign access to Chinese models, including open weight models. For Silicon Valley, this is a potentially drastic update. Chinese open source models are quick, cheap, and effective, and played an important role in getting Hugging Face under control when US proprietary models proved incapable or too expensive. US companies increasingly rely on open source models, including from the PRC, to maintain a competitive edge in a rapidly changing marketplace. 

The key challenge for US companies is that Chinese models are increasingly used in simultaneously perpetrating and defending against cyber attacks. This means that we are entering a brave new world for corporate security teams in which US companies paradoxically believe that they need to use Chinese models to deflect cyber attacks that also originate from the PRC. 

Moreover, if the CCP is concerned about the safety and security of PRC open weight models, then why do US private companies seemingly not share the same concerns? US companies are increasingly keen to use a platform that is not only ultimately answerable to the CCP, but that the CCP itself is looking to restrict out of concern for China’s own data security. For many companies, the short-term business case outweighs both data security risks related to using models to handle potentially sensitive company information and regulatory risks of engaging with PRC-based companies that could end up on a future US government restriction list. US companies therefore need to both think carefully about how and why they use PRC open-weight models, and the political risks that could come from publicly supporting such models.

Analysis

Last month, the FBI took legal control of 13 websites that it alleges are Chinese intelligence operations posing as Western consulting firms. One of the websites had cloned the site of a real company: Horizzen, a small Brisbane-based consultancy. The fake Horizzen lifted the real Horizzen’s name, address, and phone number wholesale and used them to advertise consulting work in defense, trade, and international relations. Jobseekers with legitimate security and foreign-policy credentials applied in good faith, some sending detailed CVs straight to the actual Horizzen, which was not hiring and had no idea the fake version existed until the FBI's seizure notice arrived. The FBI traced several seized domains to global proxy locations, including Catalyst Global Solutions, advertised itself as “among DC's top firms,” while its real address was traced to Lahore, Pakistan, and a third ran job ads for Australian-based positions but posted them from Thailand. Even with foreign web addresses, the FBI tied the organization to Chinese spies by tracking their crypto payments, shared digital footprints, and specific targets. Chinese intelligence has turned fake consulting job postings into a mechanism for collecting data on individuals of interest and for later potential unwitting or witting recruitment. It is important to note that thus far, there is no evidence that these operations resulted in any recruitments.

Horizzen's own reaction of dismissing its impersonation as a "cheap knock-off" for months was insufficient. The operation shows how urgent it is for companies to aggressively monitor for domain spoofing and brand impersonation rather than writing off fake postings as mere scams. Corporate inaction inadvertently allows foreign intelligence services to use legitimate firm names as cover, and this not only harms the real firms, but can damage national security and the professionals the operations target. 

This technique is part of a pattern intelligence services have been describing for years and which they made explicit in a rare joint warning last month. The Five Eyes intelligence alliance, comprising the FBI, MI5, and the domestic intelligence agencies of Australia, Canada, and New Zealand, said Chinese military intelligence officers are posing as recruiters and consultants for credible-looking front companies to target people with access to classified or privileged information. After making first contact, the fake recruiters escalate their requests: a paid report on an open-source topic, then a slightly more specific one, then a request that moves off email and onto an encrypted app. Recruits are paid from a range of a few hundred to a few thousand dollars per report. Crucially, the target list is not limited to clearance holders. Agencies such as FBI, MI5, ASIO, CSIS, and NZSIS named academics, journalists, and think-tank staff as targets, people  not accustomed or trained to think of themselves as intelligence targets.

The domain seizures rest on affidavits and civil forfeiture warrants. The PRC rejects the accusation as a US smear campaign, which is its typical response when caught spying or hacking and echoes language it used to reject similar claims by other Five Eyes members last month. With just over a month before the next Trump-Xi meeting, Beijing will characterize such accusations as a negotiating tactic or an effort by US hardliners to derail the negotiations.

Obviously, not all consulting inquiries are espionage, but caution should be exercised when engaging with new and unknown requests and expressions of interest. Employees with access to commercially or politically sensitive information should verify a consultancy before accepting paid work. This can be done the same way one would vet an unfamiliar recruiter: check the domain's history, confirm that the firm's physical address matches its claimed location, and treat a fast move to encrypted messaging or unusually generous pay for basic research as a signal worth reporting.