Project summary

ChinaTalk publishes a twice-weekly newsletter and podcast on China, technology, and US-China relations to an audience of over 35,000 which includes the most important global policymakers in the space.\

We are looking for support to increase the rigor, quality, and frequency of our coverage around China and AI safety, the drivers and contours of US-China tech competiton and risk of great power conflict.

What are this project's goals and how will you achieve them?

“AI alignment is what we want. Alignment with what? Alignment with human values. Whose human values?” ~ Marc Andreessen, June 21, 2023

AI alignment work is our best approach to manage the catastrophic risk associated with the development of artificial general intelligence (AGI). The question of values in the above quote refers to the capacity to train the large language models (LLMs) developed by private companies in the United States and elsewhere in values that represent our society. Inside the US’s AGI development race, we’ll see various value systems included or excluded from LLMs — and it will remain difficult to predict which of these systems will produce the most efficient path to AGI in the future.

There is one value system, however, that has a 100% chance of being trained into cutting-edge LLMs over the next five years: the value system of the Chinese Communist Party under Xi Jinping.

No other government has such as clearly stated and urgent desire to produce an AI competitor trained to align with the society that created it — underscoring the urgency with which the United States must understand the Chinese Communist Party and the values it will train into next-generation LLMs.

To understand the dynamics of the emerging AI competitive landscape, one must understand three interconnected phenomena: China’s AI development, China’s governance, and the US-China relationship.

Given that these three key concepts are central to the work that ChinaTalk has been engaged in over the last five years, our outfit is uniquely suited to address the needs of policymakers, technologists, government officials, as well as the best interests of the general public.

Our work in this area can be divided into five content-creation categories:

1. Fostering a deep, critical, and unbiased understanding of China’s governance model, as informed by the US government, media, and academia, as well as key leaders in US-based technology businesses;
2. Creating a definitive, English-language record of progress within China’s technology businesses and military-civil apparatus which tracks and records the creation of foundational AI technologies and systems;
3. Continuing to monitor, comment on, and analyze the US-China relationship and the risk of catastrophic conflict inherent in both China’s rise and the shift away from unipolar US dominance;
4. Cultivating an understanding of how the historical record of Cold War conflict can inform today’s policymakers on escalation and catastrophic risk — especially escalation led by the introduction of novel technologies into conflict;
5. And addressing the knowledge gaps and misconceptions that have come to dominate US-centric media on the deployment of AI technologies within China for the purposes of authoritarian control.

Why ChinaTalk?

ChinaTalk is not your typical institution when it comes to covering these subjects. We’re a media outfit, not a think-tank or university.

While acknowledging that think tanks will play a critical role in reducing future existential risks, our approach gives us tools and leverage that are absent from other organizations in this space.

In particular:

1. ChinaTalk is a “vitamin,” not a “painkiller”

Many of the organizations who read ChinaTalk analyze the China space because they are addressing acute pain points in the navigation of the geopolitical relationship. They develop expertise and hypotheses on subjects that the policy- and decision-makers deem most important in any given legislative session or fiscal quarter.

ChinaTalk, by virtue of being a media company, seeks to foster the weekly education of these same individuals, while also avoiding siloing-off the knowledge necessary to understanding.

Creating timely, relevant, and relentless content allows our organization to maintain a cadence of education within both Washington and Silicon Valley that serves to enhance the popular understanding of the aforementioned five areas of interest.

Offering a platform for experts to bring forward generalizable knowledge in an engaging and clear manner allows the technology and policy ecosystem to flourish, resist catastrophic misunderstandings and falsehoods, and ultimately serve our global community’s future.

2. ChinaTalk publishes at (or close to) the speed of AI development

Too many organizations in our space create polished quarterly reports and “definitive” documents, aiming to create reference points that will last for years or even decades. This approach is necessary and effective in areas where consistent policy is critical to geopolitical stability. For instance, a global community that changed its approach to trade policy every six weeks would severely curtail businesses reliant on this stability to flourish.

But AI development is not progressing at a speed where this type of approach can reliably help to educate and inform. The Cambrian explosion of interest, capital, and energy in AI can be understood only through diligent reporting that reflects the tempo of the industry — and traditional academia and most think tanks are simply ill-suited to cover and contextualize at the tempo that AI demands.

3. ChinaTalk optimizes accessibility without sacrificing rigor

ChinaTalk exists to enrich the elite public discourse of policymakers, journalists, technologists, and students. We seek to inform ever-growing numbers of the English-speaking public in a way that creates “on ramps” into the conversation. We seek to create content that challenges our audience weekly to grow their understanding of these issues, and we do so with compelling, public-facing content that is simplified but not simple.\

How will this funding be used?

Core Content Project: $150k

Our core project’s primary goal is to enrich public discourse at the intersection of US-China relations and AI safety. We believe that fostering global understanding of Chinese AI developments and views on AI safety is crucial for promoting the sort of global coordination that will be necessary to keep us all safe.

We aim to focus our coverage on three specific areas to achieve our project’s goals:

1. China’s nascent AI safety discourse within firms and in civil society: Over the past six months, China has begun to reckon with both the possibilities and the dangers of advances in AI. On the one hand, AI offers China an economic path forward with consistently high growth, as experienced after the Deng Xiaoping reforms of the 1980s. The need for a return to high growth is highlighted, for example, by recent statistics which show that one in five Chinese youth (aged sixteen to twenty-four) are unemployed. On the other hand, however, the safety risks that AI pose are nontrivial — and today’s businesses in China are generally ill-equipped to handle them. For instance, as mentioned on ChinaTalk earlier this year, one Chinese tech company named the night watchman as its official cybersecurity representative.
2. Furthermore, the scope of “AI safety” as it pertains to China is still forming. Discussions along AGI risk, misinformation management, and employment impacts are still in very early stages. By exploring how the discussion is changing to an English-speaking audience, we hope to better educate the world on the prospects for global dialogue and regulation on AI safety.
3. As much as possible, we hope to interview AI researchers, policy analysts, students, and entrepreneurs in China to give human perspectives to this evolving story. We will also report on perspectives and case studies from Chinese social media, where robust debates over AI safety issues like labor protection, ethics, and existential risk have already begun.
4. Developments in Chinese firms’ AI capabilities: Following ChatGPT’s release, Chinese firms have been racing to develop their own large language models (LLM). As ChinaTalk has reported, an indigenous Chinese firm has yet to produce an LLM as competitive as ChatGPT — the rollout of Baidu’s “ERNIE” in March was a flop — but innovation is taking pace at breakneck speed, and there’s no saying what a Chinese firm may be able to produce over the coming months and years.
5. The Chinese government’s response to AI: In late April 2023, the State Council, China’s senior-most governing body chaired by Xi himself, met to discuss the potential implications of AGI. The significance of advancements in AI has been on Xi’s mind for years now — and at least since 2017, when he declared that China needs to be the world’s “science and technology superpower” and introduced government “focus tasks” to achieve that end via explicitly supporting AI innovation and industry.

To explore how the Chinese government is approaching AI, we will conduct open-source research to map out how thinking among Beijing’s leaders has evolved since AI entered mainstream consciousness. Paying particular attention to how Xi Jinping approaches technology in his personal ideology, we will seek to understand how advanced AI may fit into his political vision in the next decade.

We will also research Chinese government funding for AI development, with special emphasis on under-covered topics like local governments, regional distribution, collaboration between public and private sectors, and the domestic talent pipeline.

As an example of previous coverage, see my interview with two AI regulation experts earlier this year (in two parts). It explores the role of the opaque and arbitrarily powerful Cyberspace Administration of China (CAC), collaboration between US and Chinese companies starting in the 1990s (and what has happened to that collaboration thirty years later), AI regulations that China has already considered, and how those regulations compare to the ones the US and EU have mulled over.

The format of our coverage will consist of selected translations and roundups, interviews with key players in China both in English and Chinese (which we will translate for our audience), as well as original feature reporting.

To measure our impact, we will track our audience size and influence in the broader discourse. Our reporting aspires to reach key decisionmakers in firms and governments, as well as to inform the broader national conversation surrounding AI.

Indicators of success will also be found in the quality of our open-source research. As we have noted, over the past few weeks the Chinese government has conducted raids of foreign businesses, expanded its counterespionage legislation, and closed off access to publicly available datasets. Reporting on China from abroad is poised to become much harder — and the expertise and experience of our editorial team in the needles in the haystack is therefore all the more scarce. The aforementioned challenges notwithstanding, we fully expect to discover and report on groundbreaking insights in the AI arena that even capable China researchers can no longer find today.

But our measurements of success are more ambitious than merely following the metrics of our own reporting. We anticipate that ChinaTalk’s coverage will engender a positive impact on global discourse surrounding AI safety and US-China relations. Indeed, decisionmakers everywhere — public or private sector, in the US, China, or elsewhere — will need a handle on at least the following trends to make smart strategic decisions:

In keeping decisionmakers apprised of these trends, our coverage will carry its weight many times over in moving the world toward a safer future. This project, then, is not only relevant to the goals of the fund(s) to which we are applying. It will be instrumental in fostering mutual understanding and international cooperation in the AI domain.

We intend to spend this money to buy Jordan out of his private corporate consulting engagements, allowing him to work on ChinaTalk full time, as well as bring on a full-time fellow for one year at $85k to research and write on developments around China and AI with a focus on safety. This fellow would be, to my knowledge, the only researcher in the English-speaking world devoted solely to covering China and AI safety.

Who is on your team and what's your track record on similar projects?

Jordan Schneider is the founder and editorial lead for ChinaTalk (part time). \

Caithrin Rintoul is ChinaTalk’s head of operations (part time).

ChinaTalk has two part-time editors, who all work approx 5 hrs/week. Nicholas Welch leads on newsletter editorial and audio editing and Irene Zhang is our chief China AI correspondent.

The twenty-first century’s catastrophic risks will be inextricably linked to China. Over the coming decades, China and the US will produce virtually all the cutting-edge AI developments, with all their attendant AI safety risks. The world has just witnessed how biorisk mismanagement in China produced a global pandemic. And US-China relations have reached a nadir not seen since the Mao era; this dip in relations has gutted the ability of the two nations’ governments — and militaries — to communicate effectively, increasing the probability of kinetic conflict between nuclear-armed superpowers.\

Meanwhile, in the entire United States today, there are no more than 700 China researchers. Within the US government, as former CIA analyst Dennis Wilder recently explained on ChinaTalk, the intelligence community’s demand for China analysts is growing faster than its ability to hire and train relevant talent, and thus relies on non-China experts (like former Middle East specialists) to fill the gaps.\

In other words, the number of China specialists is severely mismatched to the importance and magnitude of the catastrophic risks in which China has a key role in addressing. (As a point of comparison: the Fed alone employs over 400 PhD economists.) Without adequate funding for and realistic pathways to careers in China analysis, suboptimal policymaking is far more likely, needlessly exacerbating catastrophic risk.

ChinaTalk seeks to close this gap, both by producing accessible and rigorous China analysis, as well as by advising and promoting the work of the rising generation of China scholars. Our impact relative to our admittedly small budget has been gratifying to our team — but it has also been indicative of how deep the dearth is for compelling, career-building, and ultimately risk-reducing China analysis.

With your support, ChinaTalk could become a leader in filling the China policy gap. Our content and programs — as outlined in this grant application — promise to make our China research even better, and to produce dozens of new China scholars who will have careers researching China and reducing catastrophic risk.

Those outcomes would make good on the recommendations of 80,000 Hours’s Stephen Clare: “to work on reducing risks from AI, bioweapons or nuclear weapons directly” and “developing expertise in one of the most important bilateral relationships (such as that between the United States and China).”

I make the broader case for investing in understanding China particularly in the US context as a highly effective means to reduce catastrophic risk in this post (a 50-minute read): https://forum.effectivealtruism.org/posts/E2BghQq9pwPgtHgiH/war-between-the-us-and-china-a-case-study-for-epistemic

For the past six years, I’ve grown from scratch what is today the leading podcast and newsletter around China and technology policy. ChinaTalk now has over 26,000 subscribers, 300,000 monthly newsletter reads, and a 140% audience growth rate in the past year. ChinaTalk is the second-most-popular general-interest China-related content outlet in the US (behind only Bill Bishop’s Sinocism, who has been running his newsletter for 13 years).

One of ChinaTalk’s unique strengths is its presentation style: while maintaining both rigor and urgency, our content is orders of magnitude more engaging than work coming out of academia or think tanks. I chose not to pursue a PhD in part because my current work frees me from constraints that would limit my work’s broader reach. ChinaTalk’s stylistic approach both increases audience engagement and underscores our ultimate optimism for the future, the challenges of AI and US-China relations notwithstanding.

And thus far, I have designed and grown ChinaTalk on only a part-time basis — prior to 2023 I held a full-time job at a think tank, and this year I have supported myself primarily with consulting projects. This funding would allow me and the ChinaTalk team to prioritize producing timely and important content for our audience around China and catastrophic risk with a particular focus on AI and the risks of great power conflict.

What follows is a selection of ChinaTalk’s output has been able to accomplish just in the past two years on a budget of $35k/yr.\

AI-adjacent China coverage:

• The defining piece on the long-term implications of America’s semiconductor export controls;
• Early coverage of China’s ChatGPT fever and how regulatory concerns could throttle China’s LLMs;
• Why Beijing wants to build AGI;
• Podcast episodes about the impact of GPT4 on US-China tech competition, how AI will impact the US-China military balance, and the impact of the Chips Act on the US semiconductor ecosystem.

Semiconductor and export control coverage:

• A long-form essay breaking down the implications of the Commerce Department’s October 2022 export controls on US-China relations;
• An interview with attorney Kevin Wolf — released within four days of Commerce’s export controls — which breaks down precisely what the extremely dense and lengthy regulations actually say;
• Roundtable discussion with the eponymous “Chip Avengers” — four veritable tech experts — addressing Beijing’s point of view on the export controls.\

US-China Conflict coverage:

• Carnegie nuclear expert Tong Zhao’s take on China’s nuclear arsenal expansion, in the wake of the invasion of Ukraine;
• The epistemics of contemplating and preparing for a Chinese invasion of Taiwan.\

Helping train the next generation of China scholars:

• A living document of advice for aspiring China watchers;
  • FWIW: ChinaTalk has a substantial readership with folks under 30, and over the past two years I’ve had at least 150 calls with students and early career professionals looking for career advice.
• CCP Open Source Reading Club: Once a month I gather around fifty China analysts from around the world to discuss Party documents.

Beyond the public education ChinaTalk does through its newsletter and podcast, Jordan Schneider also meets regularly with senior US officials to discuss and advise on US-China relations and technology policy. I can share a private list with potential funders upon request of the officials I have briefed over the past year.

What are the most likely causes and outcomes if this project fails? (premortem)

Given that I have seven years of experience making engaging and impactful content around China and technology, I'm pretty confident that ChinaTalk would only gain by my ability to spend more time on the work that feeds into the platform.

Perhaps the greatest risk would be in hiring the fellow. The venn diagram of being a having strong chinese language skills, a basic understanding of the Chinese government, an interest in AI, and the ability to write well doesn't leave a ton of people with all the prerequisites for success. However, I have run internship programs in past day jobs and gone through multiple hiring cycles for part-time employees in the past, and am pretty confident that the right fellow is out there for this role.

What other funding is are you or your project getting?

We currently have no outside funding. Income is through substack subscriptions and the occasional ad buy.

ChinaTalk for its existence has thus far run as a sole proprietorship under Jordan. For any future philanthropic support, we plan to partner with FPRI, the Foreign Policy Research Institute, a respected Philadelphia-based think tank I have worked with in the past.