<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[Calvin]]></title><description><![CDATA[Calvin on AI, Technology, Society, Philosophy, Economics, and Food]]></description><link>https://blog.cal.vin</link><image><url>https://substackcdn.com/image/fetch/$s_!AglF!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb5ac185f-a363-4144-9bb9-fc5b1511b0e7_144x144.png</url><title>Calvin</title><link>https://blog.cal.vin</link></image><generator>Substack</generator><lastBuildDate>Thu, 30 Apr 2026 10:13:51 GMT</lastBuildDate><atom:link href="https://blog.cal.vin/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[Calvin Li]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[wave@cal.vin]]></webMaster><itunes:owner><itunes:email><![CDATA[wave@cal.vin]]></itunes:email><itunes:name><![CDATA[Calvin Li]]></itunes:name></itunes:owner><itunes:author><![CDATA[Calvin Li]]></itunes:author><googleplay:owner><![CDATA[wave@cal.vin]]></googleplay:owner><googleplay:email><![CDATA[wave@cal.vin]]></googleplay:email><googleplay:author><![CDATA[Calvin Li]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[The AGI race is an all‑pay auction. That’s why “over‑investment” is rational.]]></title><description><![CDATA[This unprecedented race has an AI bubble built in by design]]></description><link>https://blog.cal.vin/p/the-agi-race-is-an-allpay-auction</link><guid isPermaLink="false">https://blog.cal.vin/p/the-agi-race-is-an-allpay-auction</guid><dc:creator><![CDATA[Calvin Li]]></dc:creator><pubDate>Thu, 30 Oct 2025 18:53:51 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!0rbF!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>When the prize is &#8220;winner&#8209;takes&#8209;all&#8221; and everyone must pay their costs whether they win or lose, you don&#8217;t get measured competition&#8212;you get value (rent) dissipation [3]. That is what contest theory calls an all&#8209;pay auction [0]. In expectation, participants spend roughly the entire value of the prize in aggregate trying to win it [1][2]. <strong>What happens when the </strong><em><strong>perceived</strong></em><strong> value of the prize is nearly infinite?</strong></p><p>For AGI&#8212;where the imagined prize is monopoly&#8209;like profits across software, science, society, the next industrial revolution, the whole fabric of human civilization&#8212;equilibrium spending is enormous by construction. In this worldview, the seemingly excessive capital allocation is rational: if you cut spending while rivals do not, you lose the race and everything you&#8217;ve already invested. Google co&#8209;founder Larry Page has allegedly asserted (as relayed by investor Gavin Baker): &#8220;<em>I am willing to go bankrupt rather than lose this race</em>&#8221; [4].</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!0rbF!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!0rbF!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png 424w, https://substackcdn.com/image/fetch/$s_!0rbF!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png 848w, https://substackcdn.com/image/fetch/$s_!0rbF!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png 1272w, https://substackcdn.com/image/fetch/$s_!0rbF!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!0rbF!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png" width="1456" height="971" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/b17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:971,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!0rbF!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png 424w, https://substackcdn.com/image/fetch/$s_!0rbF!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png 848w, https://substackcdn.com/image/fetch/$s_!0rbF!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png 1272w, https://substackcdn.com/image/fetch/$s_!0rbF!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb17d8219-5190-4df6-9286-8127ab8e349a_1536x1024.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>These massive investments are well known: Microsoft told investors it expects capital expenditures (capex) to exceed <strong>$30&#8239;billion</strong> in a single quarter [5], while Alphabet lifted its <strong>2025</strong> capex plans to <strong>~$85&#8239;billion</strong> [5]. McKinsey maps a <strong>~$6.7&#8211;$7&#8239;trillion</strong> global data&#8209;center build through <strong>2030</strong> [6]. Nvidia&#8217;s Blackwell&#8209;class chips are <strong>$30k&#8211;$40k</strong> apiece [7]. OpenAI&#8217;s Broadcom deal targets <strong>10&#8239;GW</strong> of custom accelerators [9]. The <strong>Stargate</strong> alliance (OpenAI&#8211;Oracle, with SoftBank) is scoped at up to <strong>$500&#8239;billion</strong> and <strong>10&#8239;GW</strong> [10]. And the <strong>IEA</strong> estimates data&#8209;center electricity demand will roughly <strong>double</strong> by <strong>2030</strong> [8].</p><p>For scale: the Manhattan Project cost roughly <strong>$2&#8239;billion</strong> in 1940s dollars (&#8776;&#8239;<strong>$30&#8211;$50&#8239;billion</strong> today) [11], Apollo <strong>$25.8&#8239;billion</strong> then (&#8776;&#8239;<strong>$250&#8211;$300&#8239;billion</strong> today) [12], and the Interstate Highway System is about <strong>$634&#8239;billion</strong> in <strong>2024</strong> dollars [13]. AGI&#8209;related capex now rivals or exceeds the total costs of these projects every year.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h1>A minimal model</h1><p>An all&#8209;pay auction is as it sounds: all bidders pay regardless of who wins. Once you have sunk enough resources into the race, the marginal cost to continue playing is small relative to the prize. If you bid <strong>$0.99</strong> for a <strong>$1.00</strong> prize, and a rival bids <strong>$1.00</strong>, it can be rational to raise by <strong>$0.02</strong> to win&#8212;even though your total outlay is now <strong>$1.01</strong> to win just <strong>$1.00</strong>. In this type of auction, the best strategy is often to not enter at all.</p><p>What makes this race look particularly like an all&#8209;pay auction is the lack of a clear definition and finish line for what AGI even is [27]. There is an exuberant belief that sufficient investment will get us there based on known scaling laws, without clarity on sufficiency and the technological roadmap. While theory suggests that group spend should approach perceived value, there lacks clarity as to what that final value looks like. In addition, real&#8209;world experiments show that people tend to over&#8209;bid relative to theory [15]. This mirrors the current tech race, where status, imperfect information, &#8220;learning while doing,&#8221; and fear&#8212;fear of missing out and being a loser&#8212;can cause participants to overshoot theory.</p><p>A common instance of an all-pay auction are the infamous <strong>penny auctions</strong>, exploiting ill-informed bidders, for example bidders have paid <strong>$7,264</strong> for a <strong>$180</strong> item [28].</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!3La4!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1603122b-2afe-4d97-a49d-067aea257a5f_1036x522.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!3La4!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1603122b-2afe-4d97-a49d-067aea257a5f_1036x522.png 424w, https://substackcdn.com/image/fetch/$s_!3La4!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1603122b-2afe-4d97-a49d-067aea257a5f_1036x522.png 848w, https://substackcdn.com/image/fetch/$s_!3La4!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1603122b-2afe-4d97-a49d-067aea257a5f_1036x522.png 1272w, https://substackcdn.com/image/fetch/$s_!3La4!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1603122b-2afe-4d97-a49d-067aea257a5f_1036x522.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!3La4!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1603122b-2afe-4d97-a49d-067aea257a5f_1036x522.png" width="1036" height="522" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/1603122b-2afe-4d97-a49d-067aea257a5f_1036x522.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:522,&quot;width&quot;:1036,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!3La4!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1603122b-2afe-4d97-a49d-067aea257a5f_1036x522.png 424w, https://substackcdn.com/image/fetch/$s_!3La4!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1603122b-2afe-4d97-a49d-067aea257a5f_1036x522.png 848w, https://substackcdn.com/image/fetch/$s_!3La4!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1603122b-2afe-4d97-a49d-067aea257a5f_1036x522.png 1272w, https://substackcdn.com/image/fetch/$s_!3La4!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1603122b-2afe-4d97-a49d-067aea257a5f_1036x522.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><h1><strong>An unprecedented moment in human history</strong></h1><p>There has never been this combination of scale, competition, and investment in fast&#8209;depreciating assets&#8212;burning through GPUs, electricity, and the time of ultra&#8209;scarce (and ultra&#8209;expensive) AI scientists&#8212;all financed primarily by private balance sheets. Unlike rail rights&#8209;of&#8209;way or overbuilt fiber that retained residual value, today&#8217;s frontier spend sits on a few&#8209;year clock&#8212;servers and GPU accelerators that turn over quickly (even as shells and interconnects endure)&#8212;while data&#8209;center power demand is set to roughly <strong>double</strong> by <strong>2030</strong> [8].</p><p>The rapid depreciation dynamics combined with the sheer scale and intense private competition makes this the largest and most unprecedented all-pay-auction in the history of humanity, with enormous implications for both our future and for the economy regardless of the outcome.</p><h1>What happens if AI leaders are correct? A bubble.</h1><p>If all the tech billionaires chasing AGI are correct&#8212;that the prize is singular (winner&#8209;takes&#8209;all or most) and that AGI is attainable&#8212;the economic situation looks ominous:</p><p><strong>Most of the value gets burned en route.</strong> In canonical all&#8209;pay contests, aggregate spend tends to the value of the prize [1]; experiments often find over&#8209;dissipation [15]. With a dozen serious AGI bidders, more than <strong>90%</strong> of ultimate value will be burned by the group in expectation [0][1]. The prize can be hot, and the bonfire can burn just as hot.</p><p><strong>Losses concentrate in ordinary savings.</strong> The losers are not just founders and a handful of VCs. Public markets, pensions, and retirement accounts are bankrolling these investments. About <strong>53.7%</strong> of U.S. households own mutual funds [16]; pensions are a massive balance&#8209;sheet item in the national accounts [17]. If all but one of the dozen major bidders lose, those losses propagate back into the vehicles most people own.</p><p><strong>AGI&#8217;s value will take significant time to pay out.</strong> There&#8217;s a difference between creating value and capturing value. There&#8217;s also a difference between inventing a technology and deploying it. Given the severity of the economic fallout, there won&#8217;t be a wellspring of capital to flow to the winner, and deployment will not be instantaneous; the world will still need to adapt to the new reality.</p><p>This is why &#8220;AI bubble&#8221; worries do <strong>not</strong> require technological failure. Contest logic alone can yield bubble&#8209;like P&amp;L: one epic winner, many deep losers, and an index that looks over&#8209;capitalized until cash flows land. Markets are already keying on capex as the hinge variable [21]; buybacks are giving way to record spending, and patience has limits.</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/the-agi-race-is-an-allpay-auction?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">If you find this insightful, feel free to share it.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/the-agi-race-is-an-allpay-auction?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://blog.cal.vin/p/the-agi-race-is-an-allpay-auction?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><h1>How the auction might end</h1><p>All&#8209;pay auctions do end. In the real world, they stop&#8212;or morph&#8212;through a few predictable doors:</p><ul><li><p><strong>The prize becomes legible.</strong> If we converge on an operational definition of &#8220;AGI&#8221; and even a clear roadmap to attaining such a thing, firms can price the end state and reduce the fog of war. When the prize is better measured, marginal bids shrink toward measured value.</p></li><li><p><strong>A decisive lead becomes common knowledge.</strong> Once one player&#8217;s edge is credible and durable, rivals&#8217; expected payoff from the next dollar collapses; they quit. The cousin model here is the <strong>war of attrition</strong>: when others are expected to quit first, the rational move is to stop before you bleed out [24].</p></li><li><p><strong>The prize fragments.</strong> If the market sustains multiple niches&#8212;foundation models vs. specialized agents; chips vs. base models vs. vertical apps&#8212;effort is spread across several &#8220;first prizes,&#8221; and aggregate waste falls. Contest theory shows prize structure matters; multiple prizes typically dissipate less value than a single jackpot [25][26].</p></li><li><p><strong>Coalitions / Mergers form.</strong> Joint ventures, chip alliances, and long&#8209;term supply pacts turn rivals into partial teammates on the costliest inputs (silicon, power). Think OpenAI&#8211;Broadcom&#8217;s <strong>10&#8239;GW</strong> accelerator program [9] and the <strong>Stargate</strong> build&#8209;out with Oracle and SoftBank [10]; these are coalition bids on the input stack rather than duplicative concrete and copper.</p></li><li><p><strong>Budget constraints bind.</strong> If revenue trails capex for long enough&#8212;or bottlenecks bite&#8212;capital markets ration the marginal dollar. Citi projects AI infrastructure capex of <strong>~$490&#8239;billion by 2026</strong> [14]; investors are already grading firms on whether spend turns into cash flow [21], with grid constraints looming large [20].</p></li><li><p><strong>Policy or physics shrink the prize.</strong> Licensing, export controls, interconnection queues, and power availability can lower the feasible return to scale. The IEA&#8217;s scenarios point to tight power fundamentals [8]; grid frictions and interconnection hurdles are mounting [20][22][23].</p></li><li><p><strong>AGI is discovered to be unattainable.</strong> If the frontier stalls, capital reprices and the contest ends abruptly. Then nobody wins and everyone, including society, loses. That possibility is precisely why coordination now is cheaper than a disorderly unwind later.</p></li></ul><h1>Can we spend smarter? Five ways to reduce waste</h1><ol><li><p><strong>Merge or cross&#8209;license earlier.</strong> If the game is truly winner&#8209;takes&#8209;all, consolidating upstream to avoid proving the same point four times is efficient. There are precedents for pre&#8209;competitive collaboration that pooled risk but kept rivalry alive downstream&#8212;<strong>SEMATECH</strong> in U.S. chips; <strong>Airbus</strong> in European aerospace [18][19].</p></li><li><p><strong>Form public&#8211;private &#8220;mission&#8221; vehicles.</strong> The scale is already approaching or exceeding traditionally nation-scale projects. Manhattan, Apollo, and the Interstate Highway System were centrally coordinated because duplication would be wasteful. Today&#8217;s AI build is already larger in many scenarios; a mission vehicle can concentrate procurement, reduce duplication, and publish interfaces that split the prize into layers [11][12][13][6].</p></li><li><p><strong>Standardize and &#8220;split the prize.&#8221;</strong> Open interfaces and clear safety baselines make it easier for multiple winners to coexist&#8212;chips here, base models there, vertical apps elsewhere&#8212;defusing the incentive to torch the commons for a single crown [25][26].</p></li><li><p><strong>Gate capex to real demand and real constraints.</strong> Tie the next tranche of GPUs or gigawatts to observed utilization, revenues, and interconnection progress. Grid operators and the IEA are already warning about the bottlenecks; investors are as well [8][20][21].</p></li><li><p><strong>Make &#8220;coopetition&#8221; boring.</strong> Long&#8209;term supply contracts, shared fabs, site&#8209;level partnerships&#8212;think <strong>Stargate&#8217;s</strong> shared footprint&#8212;move competition up the stack toward product differentiation and away from duplicating fixed assets [10].</p></li></ol><h1>Conclusion: It&#8217;s rational for them to risk a recession for all of us</h1><p>One <strong>spectacular</strong> winner, many <em>deep</em> losers, and the losses that reverberates into the rest of the economy. This is the consequences of the current structure of the AGI race: an all-pay single jackpot auction that is proven to expect <strong>massive rent dissipation</strong>. This is bubble-by-design, even if the technology proves to be everything the AI leaders claim. The firms are acting in their <strong>rational self interest</strong> and as predicted by contest theory, so we cannot just ask them to change their behavior. The only way to get them to steer clear of economic catastrophe is if the game changes or we change the game.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/the-agi-race-is-an-allpay-auction?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://blog.cal.vin/p/the-agi-race-is-an-allpay-auction?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/the-agi-race-is-an-allpay-auction/comments&quot;,&quot;text&quot;:&quot;Leave a comment&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://blog.cal.vin/p/the-agi-race-is-an-allpay-auction/comments"><span>Leave a comment</span></a></p><h1>Sources</h1><p>[0] <em>All&#8209;pay auction</em> (Wikipedia) &#8211; overview and intuitive framing.<a href="https://en.wikipedia.org/wiki/All-pay_auction?utm_source=chatgpt.com"> Wikipedia<br></a>[1] Baye, Kovenock &amp; de Vries (1996), &#8220;The All&#8209;Pay Auction with Complete Information,&#8221; <em>AER</em> &#8211; formal equilibrium results and dissipation.<a href="https://digitalcommons.chapman.edu/economics_articles/121/?utm_source=chatgpt.com"> Digital Commons<br></a>[2] Skaperdas (1996), &#8220;Contest Success Functions,&#8221; <em>Economic Theory</em> &#8211; how prize structure and effort allocation work.<a href="https://link.springer.com/article/10.1007/BF01213906?utm_source=chatgpt.com"> SpringerLink<br></a>[3] <em>Rent&#8209;seeking</em> (Wikipedia) &#8211; intuitive treatment of rent dissipation.<a href="https://en.wikipedia.org/wiki/Rent-seeking?utm_source=chatgpt.com"> Wikipedia<br></a>[4] Yahoo Finance (2024), citing investor Gavin Baker relaying Larry Page&#8217;s alleged line.<a href="https://finance.yahoo.com/news/thinking-selling-nvidia-stock-larry-093000946.html?utm_source=chatgpt.com"> Yahoo Finance<br></a>[5] Reuters (July&#8239;2025): Microsoft &gt;$30B quarterly capex; Alphabet lifts 2025 capex to ~$85B.<a href="https://www.reuters.com/business/media-telecom/meta-shares-jump-ai-fuels-ad-sales-outweighing-big-capital-costs-2025-07-30/"> Reuters<br></a>[6] McKinsey (2025), &#8220;The cost of compute: a $7T race to scale data centers.&#8221;<a href="https://www.mckinsey.com/industries/technology-media-and-telecommunications/our-insights/the-cost-of-compute-a-7-trillion-dollar-race-to-scale-data-centers?utm_source=chatgpt.com"> McKinsey &amp; Company<br></a>[7] Reuters (Mar&#8239;2024): Nvidia Blackwell pricing $30k&#8211;$40k.<a href="https://www.reuters.com/technology/nvidias-new-ai-chip-be-priced-over-30000-cnbc-reports-2024-03-19/"> Reuters<br></a>[8] IEA, <em>Electricity 2025</em> / &#8220;Data centres and AI&#8221; &#8211; data&#8209;center demand roughly doubling by 2030.<a href="https://www.iea.org/news/ai-is-set-to-drive-surging-electricity-demand-from-data-centres-while-offering-the-potential-to-transform-how-the-energy-sector-works?utm_source=chatgpt.com"> IEA<br></a>[9] Reuters (Oct&#8239;2025): OpenAI&#8211;Broadcom to deploy 10&#8239;GW of custom accelerators.<a href="https://www.reuters.com/business/openai-taps-broadcom-build-its-first-ai-processor-latest-chip-deal-2025-10-13/?utm_source=chatgpt.com"> Reuters<br></a>[10] Reuters (Oct&#8239;2025): Stargate (OpenAI&#8211;Oracle, with SoftBank) up to $500B / 10&#8239;GW.<a href="https://www.reuters.com/business/openai-oracle-announce-stargate-data-center-site-wisconsin-2025-10-22/?utm_source=chatgpt.com"> Reuters<br></a>[11] National Park Service / CMU: Manhattan Project cost (~$2B then; $30&#8211;$50B today).<a href="https://ethos.lps.library.cmu.edu/article/id/35/?utm_source=chatgpt.com"> ethos.lps.library.cmu.edu<br></a>[12] Planetary Society: Apollo (~$25.8B then; $257B 2020 dollars).<a href="https://www.planetary.org/space-missions/apollo?utm_source=chatgpt.com"> Planetary Society<br></a>[13] <em>Interstate Highway System</em> (Wikipedia): ~$634B in 2024 dollars.<a href="https://en.wikipedia.org/wiki/Interstate_Highway_System?utm_source=chatgpt.com"> Wikipedia<br></a>[14] Reuters (Sept&#8239;2025): Citi projects AI infra capex $2.8T by 2029; ~$490B by 2026.<a href="https://www.reuters.com/world/china/citigroup-forecasts-big-techs-ai-spending-cross-28-trillion-by-2029-2025-09-30/?utm_source=chatgpt.com"> Reuters<br></a>[15] Gneezy &amp; Smorodinsky (2006), &#8220;All&#8209;pay auctions&#8212;an experimental study.&#8221;<a href="https://rady.ucsd.edu/_files/faculty-research/uri-gneezy/all-pay-auctions.pdf?utm_source=chatgpt.com"> Rady School of Management<br></a>[16] ICI (2024): 53.7% of U.S. households own mutual funds.<a href="https://www.ici.org/files/2024/per30-08.pdf?utm_source=chatgpt.com"> ICI<br></a>[17] Federal Reserve, Z.1 Financial Accounts (Table L.229): pension entitlements are a large household asset.<a href="https://www.federalreserve.gov/releases/z1/20250612/html/l229.htm?utm_source=chatgpt.com"> Federal Reserve<br></a>[18] CSIS (2023): SEMATECH lessons for pre&#8209;competitive consortia under the CHIPS Act.<a href="https://www.csis.org/analysis/implementing-chips-act-sematechs-lessons-national-semiconductor-technology-center?utm_source=chatgpt.com"> CSIS<br></a>[19] <em>Airbus</em> (Wikipedia): origins as a European industrial consortium.<a href="https://en.wikipedia.org/wiki/Airbus?utm_source=chatgpt.com"> Wikipedia<br></a>[20] S&amp;P Global Market Intelligence (Oct&#8239;2025): grid congestion and interconnection bottlenecks.<a href="https://www.spglobal.com/market-intelligence/en/news-insights/research/2025/10/grid-congestion-remains-key-issue-as-data-center-load-growth-stresses-system?utm_source=chatgpt.com"> S&amp;P Global<br></a>[21] Reuters (Oct&#8239;27,&#8239;2025): buybacks take a back seat as record AI capex stretches investor patience.<a href="https://www.reuters.com/business/finance/buybacks-take-backseat-ai-drives-record-us-capex-spending-2025-10-27/?utm_source=chatgpt.com"> Reuters<br></a>[22] Reuters (Apr&#8239;2025): Google&#8211;PJM collaboration to use AI for faster interconnection.<a href="https://www.reuters.com/technology/google-deploys-ai-speed-up-connections-pjm-largest-us-power-grid-2025-04-10/?utm_source=chatgpt.com"> Reuters<br></a>[23] Utility Dive (Sep&#8239;2025): PJM stakeholder debates on data&#8209;center demand response and curtailment.<a href="https://www.utilitydive.com/news/pjm-stakeholders-ncbl-data-center-fast-track/759096/?utm_source=chatgpt.com"> Utility Dive<br></a>[24] <em>War of attrition</em> (Wikipedia): &#8220;others quit first&#8221; logic.<a href="https://en.wikipedia.org/wiki/War_of_attrition_%28game%29?utm_source=chatgpt.com"> Wikipedia<br></a>[25] Barut &amp; Kovenock (1998), &#8220;The symmetric multiple&#8209;prize all&#8209;pay auction.&#8221;<a href="https://www.sciencedirect.com/science/article/abs/pii/S0176268098000299?utm_source=chatgpt.com"> ScienceDirect<br></a>[26] Faravelli (2012), &#8220;Single vs. multiple&#8209;prize all&#8209;pay auctions to finance public goods.&#8221;<a href="https://ideas.repec.org/a/eee/jeborg/v81y2012i2p677-688.html?utm_source=chatgpt.com"> IDEAS/RePEc<br></a>[27] <em>A Definition of AGI</em> (2025)<a href="https://arxiv.org/abs/2510.18212"> Arxiv<br></a>[28] Penny Auctions - How to sell a $180 tablet for $7,264.<a href="https://www.curiousgnu.com/penny-auctions"> Curious Gnu</a></p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Calvin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Explanation of The Stranger, by Jean-Paul Sartre | Original English Translation]]></title><description><![CDATA[Jean-Paul Sartre, &#8220;Explanation of The Stranger&#8221;, in Situations I, 1947]]></description><link>https://blog.cal.vin/p/explanation-of-the-stranger-by-jean</link><guid isPermaLink="false">https://blog.cal.vin/p/explanation-of-the-stranger-by-jean</guid><dc:creator><![CDATA[Calvin Li]]></dc:creator><pubDate>Sat, 15 Feb 2025 17:54:46 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!ktt7!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!ktt7!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!ktt7!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp 424w, https://substackcdn.com/image/fetch/$s_!ktt7!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp 848w, https://substackcdn.com/image/fetch/$s_!ktt7!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp 1272w, https://substackcdn.com/image/fetch/$s_!ktt7!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!ktt7!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp" width="1024" height="1024" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1024,&quot;width&quot;:1024,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;An oil painting portrait of Albert Camus in the style of 1940s French art, designed for an article featuring an English translation of Jean-Paul Sartre's 'Explanation of the Stranger.' Camus is depicted in a three-quarter view with a contemplative, introspective expression and expressive, loose brush strokes. The color palette consists of muted, earthy tones, including warm browns, soft grays, and hints of melancholic blue. The background is an abstract, blurred depiction of a post-war Parisian scene, evoking the intellectual vibrancy and reflective mood of the era. Subtle symbolic elements, such as abstract forms or shadowy figures, allude to existentialism and the theme of the outsider, nodding to both Camus' famous work and Sartre's commentary.&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="An oil painting portrait of Albert Camus in the style of 1940s French art, designed for an article featuring an English translation of Jean-Paul Sartre's 'Explanation of the Stranger.' Camus is depicted in a three-quarter view with a contemplative, introspective expression and expressive, loose brush strokes. The color palette consists of muted, earthy tones, including warm browns, soft grays, and hints of melancholic blue. The background is an abstract, blurred depiction of a post-war Parisian scene, evoking the intellectual vibrancy and reflective mood of the era. Subtle symbolic elements, such as abstract forms or shadowy figures, allude to existentialism and the theme of the outsider, nodding to both Camus' famous work and Sartre's commentary." title="An oil painting portrait of Albert Camus in the style of 1940s French art, designed for an article featuring an English translation of Jean-Paul Sartre's 'Explanation of the Stranger.' Camus is depicted in a three-quarter view with a contemplative, introspective expression and expressive, loose brush strokes. The color palette consists of muted, earthy tones, including warm browns, soft grays, and hints of melancholic blue. The background is an abstract, blurred depiction of a post-war Parisian scene, evoking the intellectual vibrancy and reflective mood of the era. Subtle symbolic elements, such as abstract forms or shadowy figures, allude to existentialism and the theme of the outsider, nodding to both Camus' famous work and Sartre's commentary." srcset="https://substackcdn.com/image/fetch/$s_!ktt7!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp 424w, https://substackcdn.com/image/fetch/$s_!ktt7!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp 848w, https://substackcdn.com/image/fetch/$s_!ktt7!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp 1272w, https://substackcdn.com/image/fetch/$s_!ktt7!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F45ce6cb8-c8fe-4250-af26-7682d4f082ac_1024x1024.webp 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>    Barely out of the presses, Camus&#8217;s The Stranger enjoyed enormous favor. It was said again and again that it was &#8220;the best book since the armistice.&#8221; In the midst of the literary production of the time, this novel was itself a stranger. It came to us from the other side of the line, from across the sea; it spoke of the sun in that bitter, coalless spring&#8212;not as an exotic wonder but with the weary familiarity of those who have indulged in it too much; it cared not to once again inter its own hands the old regime, nor to instill in us the sense of our own unworthiness; while reading, one recalled that there had once been works which claimed to be self-justifying and to prove nothing. Yet in exchange for this very gratuitousness the novel remained rather ambiguous: how were we to understand this character, who, the day after his mother&#8217;s death, &#8220;took baths, began an illicit liaison and went to laugh at a comedy film,&#8221; who killed an Arab &#8220;because of the sun&#8221; and who, on the eve of his capital execution&#8212;affirming that he &#8220;had been happy and still was&#8221;&#8212;desired a multitude of spectators around the scaffold &#8220;to greet him with cries of hate, of hate&#8221;? Some said, &#8220;He&#8217;s a simpleton, a poor fellow&#8221;; others, with more insight, declared, &#8220;He is an innocent.&#8221; And yet one could not grasp the meaning of that innocence.</p><p>    Mr. Camus, in The Myth of Sisyphus (published a few months later), gave us the exact commentary on his work: his hero is neither good nor evil, neither moral nor immoral. These categories do not suit him; he belongs to a very singular species which the author designates as &#8220;absurd.&#8221; But under Mr. Camus&#8217;s pen that very word takes on two very different meanings: the absurd is both a state of affairs and the lucid awareness certain persons have of that state. Is &#8220;absurd&#8221; the man who, from a fundamental absurdity, infallibly draws the conclusions that are imposed? Here one witnesses the same shift in meaning as when one calls &#8220;swing&#8221; a youth who dances swing. Then what is the absurd as a state of affairs, as an original datum? Nothing less than the relation of man to the world. The primary absurdity manifests itself first as a divorce: the divorce between man&#8217;s aspirations toward unity and the insurmountable dualism of mind and nature, between man&#8217;s thrust toward the eternal and the finite nature of his existence, between the very &#8220;concern&#8221; that is his essence and the futility of his efforts. Death, the irreducible pluralism of truths and beings, the unintelligibility of the real, chance&#8212;these are the poles of the absurd. Admittedly, these are not altogether new themes, and Mr. Camus does not present them as such. They were already enumerated, as early as the seventeenth century, by a certain kind of dry, terse, and contemplative reason that is peculiarly French: they served as commonplaces for classical pessimism. Was it not Pascal who insisted on &#8220;the natural misery of our weak, mortal condition&#8212;so wretched that nothing can console us when we scrutinize it&#8221;? Is it not he who stakes his claim for reason? Would he not unreservedly approve of Mr. Camus&#8217;s remark, &#8220;The world is neither (quite) rational nor to that extent irrational&#8221;? Does he not show us that &#8220;custom&#8221; and &#8220;diversion&#8221; conceal in man &#8220;his nothingness, his abandonment, his insufficiency, his impotence, his void&#8221;?</p><p>    By the icy style of The Myth of Sisyphus and the subject of his essays, Mr. Camus places himself within the grand tradition of those French moralists whom Andler rightly calls the precursors of Nietzsche; and regarding the doubts he raises about the reach of our reason, they lie in the more recent tradition of French epistemology. When one thinks of scientific nominalism, of Poincar&#233;, Duhem, Meyerson, one better understands the reproach our author levels against modern science: &#8220;&#8230; You speak to me of an invisible planetary system in which electrons revolve around a nucleus. You explain this world to me with an image. I then recognize that you have come to poetry&#8230;&#8221;</p><p>    That is what is expressed, almost simultaneously, by an author who draws from the same sources when he writes: &#8220;(Physics) employs indiscriminately mechanical, dynamic or even psychological models, as though, freed from ontological pretensions, it became indifferent to the classical antinomies of mechanism or dynamism&#8212;which presuppose a nature in itself.&#8221; Mr. Camus is somewhat coy in citing texts by Jaspers, by Heidegger, by Kierkegaard&#8212;authors he does not seem, moreover, to fully understand. Yet his true masters lie elsewhere: in the twist of his reasonings, the clarity of his ideas, the cut of his essayistic style, and a certain kind of orderly, ceremonious, desolate, sunlit severity&#8212;all of which herald a classic, a Mediterranean.</p><p>    Not even his method (&#8220;it is the balance of evidence and lyricism that alone can grant us simultaneous access to both emotion and clarity&#8221;) fails to evoke the ancient &#8220;passionate geometries&#8221; of Pascal or Rousseau, and to bring him closer to Maurras&#8212;this other Mediterranean, from whom he still differs in so many respects, much more than from any German phenomenologist or Danish existentialist.</p><p>    But Mr. Camus would, no doubt, gladly concede all this. In his eyes his originality lies in taking his ideas to their ultimate end: for him it is not a question of collecting pessimistic maxims. Granted, the absurd is not in man nor in the world when taken separately; yet since it is man&#8217;s very condition to be &#8220;being-in-the-world,&#8221; the absurd, in the final analysis, is one with the human condition. Hence it is not primarily the object of a mere notion&#8212;it is a desolate illumination that reveals it to us.</p><p>    &#8220;Waking, tram, four hours at the office or factory, meal, tram, four hours of work, meal, sleep, and then Monday, Tuesday, Wednesday, Thursday, Friday, Saturday on the same rhythm&#8230;&#8221; and then, all at once, &#8220;the scenery collapses&#8221; and we are plunged into a hopeless lucidity. So if we know how to refuse the deceptive succor of religions or existential philosophies, we cling to a few essential certainties: the world is chaos&#8212;a &#8220;divine equivalence born of anarchy&#8221;; there is no tomorrow, for we die.</p><p>    &#8220;&#8230; In a universe suddenly deprived of illusions and light, man feels like a stranger. This exile is without remedy, for he is deprived both of the memories of a lost homeland and of the hope of a promised land.&#8221; For indeed man is not the world: &#8220;If I were a tree among trees&#8230; this life would have meaning&#8212;or rather this problem would cease to exist, for I would be part of that world. I would be that world, to which I now oppose myself with all my consciousness&#8230;&#8221;</p><p>    Thus is partially explained the title of our novel: the stranger is the man set against the world; Mr. Camus might just as well have chosen, to designate his work, the title of one of Georges Gissing&#8217;s works: Born in Exile. The stranger is also the man among men. &#8220;There are days when&#8230; one finds, as though a stranger, the one one had loved.&#8221; And finally, it is myself in relation to myself&#8212;in other words, the man of nature vis-&#224;-vis the mind: &#8220;The stranger who, at certain moments, comes to meet us in the mirror.&#8221;</p><p>    But that is not all: there is also a passion for the absurd. The absurd man does not commit suicide; he wishes to live, without renouncing any of his certainties, without a tomorrow, without hope, without illusion, and without resignation either. The absurd man asserts himself in revolt. He fixes his gaze on death with a passionate intensity, and that fascination liberates him: he understands the &#8220;divine irresponsibility&#8221; of the condemned man. All is permitted, for God is not, and we all must die. All experiences are equivalent; one must only acquire as many as possible.</p><p>    &#8220;The present and the succession of presents before a soul that is ever aware&#8212;this is the ideal of the absurd man.&#8221; All values crumble before this &#8220;ethic of quantity&#8221;; the absurd man, hurled into this world&#8212;revolt, irreverence&#8212;has &#8220;nothing to justify.&#8221; He is innocent. Innocent like those primitives spoken of by S. Maugham before the arrival of the pastor who instructs them in Good and Evil, in what is permitted and what is forbidden: for him everything is allowed. Innocent in every sense of the word&#8212;indeed, a fool as well, if you will. And it is only in this way that we fully understand the title of Camus&#8217;s novel. The stranger he wishes to depict is precisely one of those terrible innocents who scandalize society by refusing to abide by its rules. He lives among strangers, yet to them too he is a stranger. That is why some will love him&#8212;like Marie, his mistress, who holds him dear &#8220;because he is odd&#8221;&#8212;and others will hate him for it, as does that jury at the trial whose hatred suddenly rises against him. And we, who, upon opening the book, are not yet acquainted with the sentiment of the absurd, would in vain try to judge him by our accustomed standards&#8212;for us too he is a stranger.</p><p>    Thus is roughly indicated the manner in which we must regard the hero of The Stranger. Had Mr. Camus wished to write a novel of thesis, it would not have been difficult for him to show a civil servant enthroned within his family, then suddenly seized by the intuition of the absurd, struggling for a moment before finally resolving to live the fundamental absurdity of his condition. The reader would have been convinced both of the character and, for the same reasons, by the character himself. Or, again, he might have recounted the life of one of those saints of the absurd&#8212;as he enumerates in The Myth of Sisyphus, who enjoy his particular favor: Don Juan, the Comedian, the Conqueror, the Creator. This is not what he has done, and even for the reader familiar with the theories of the absurd, Meursault&#8212;the hero of The Stranger&#8212;remains ambiguous. Certainly we are assured that he is absurd, and ruthless lucidity is his principal trait. Moreover, in more than one respect he is constructed in such a manner as to serve as a confirmed illustration of the theories expounded in The Myth of Sisyphus. For example, Mr. Camus writes in that work: &#8220;A man is more a man by the things he keeps silent about than by the things he says.&#8221; And Meursault is an example of that virile silence, that refusal to indulge in empty verbiage: &#8220;(When asked) if he had noticed that I was withdrawn, he only acknowledged that I did not speak for the sake of speaking.&#8221; And just two lines above, the same witness declared that Meursault &#8220;was a man.&#8221; &#8220;(When asked) what he meant by that, he stated that everyone knew what he meant.&#8221;</p><p>    Similarly, Mr. Camus elaborates at length on love in The Myth of Sisyphus: &#8220;We call love that which binds us to certain beings only by reference to a collective way of seeing for which books and legends are responsible.&#8221; And, concurrently, we read in The Stranger: &#8220;She then wanted to know if I loved her. I replied&#8230; that it meant nothing, but that perhaps I did not love her.&#8221; From this point of view, the debate that ensues at the trial and in the mind of the reader concerning the question &#8220;Did Meursault love his mother?&#8221; is doubly absurd. First, as the lawyer remarks, &#8220;Is he accused of having buried his mother or of having killed a man?&#8221; But above all, the word &#8220;love&#8221; is rendered meaningless. Sure enough, Meursault placed his mother in an asylum&#8212;because he lacked money and because they &#8220;had nothing more to say to one another.&#8221; Surely, he did not visit her often, &#8220;because it consumed his Sunday&#8212;not to mention the effort to catch the bus, buy tickets, and drive for two hours.&#8221; But what does that signify? Is it not all confined to the present, entirely determined by his immediate whims? What we call a sentiment is nothing but the abstract unity and significance of discontinuous impressions. I do not always think of those I love; yet I claim that I love them even when they do not occupy my thoughts&#8212;and I would be willing to compromise my tranquility in the name of an abstract sentiment in the absence of any genuine, immediate emotion.</p><p>    Meursault thinks and acts differently: he does not wish to know those grand, continuous feelings, all alike; for him, love does not exist&#8212;not even loves. Only the present matters, the concrete. He goes to see his mother when he feels like it, that is all. If the desire is there, it will be strong enough to make him catch the bus; if another concrete desire arises, it will be forceful enough to have him run at full speed and leap into a moving truck. Yet he always designates his mother with the tender, childlike word &#8220;maman&#8221; and never misses an opportunity to understand and identify with her. &#8220;Of love, I know only that mixture of desire, tenderness and understanding which binds me to a certain being.&#8221; Thus one sees that one must not neglect the theoretical side of Meursault&#8217;s character.</p><p>    Likewise, many of his adventures primarily serve to highlight one or another aspect of fundamental absurdity. For instance, as we have seen, The Myth of Sisyphus extols the &#8220;perfect availability of the condemned man, before whom the prison gates open at a certain small dawn&#8221;&#8212;and it is for us to enjoy that dawn and that availability that Mr. Camus has condemned his hero to the death penalty. &#8220;How had I not seen,&#8221; he is made to say, &#8220;that nothing was more important than an execution&#8230; and, in a sense, it was even the only thing truly interesting for a man!&#8221; One could multiply examples and quotations. And yet this lucid, indifferent, taciturn man is not entirely constructed for the cause&#8217;s sake. No doubt, once sketched, his character completed itself&#8212;the character perhaps had a certain inherent heft. In any event, his absurdity does not seem to be a conquest but a given: he is just that way, period. He will have his moment of illumination on the final page, but he has always lived according to Mr. Camus&#8217;s standards. If there were any grace in the absurd, one should say that he possesses it. He does not seem to pose any of those questions that are raised in The Myth of Sisyphus; one does not see that he was in revolt before being condemned to death. He was happy, he allowed himself to drift, and his happiness does not even seem to have known that secret bite&#8212;which Mr. Camus repeatedly points out in his essay&#8212;that comes from the dazzling presence of death. His indifference often appears merely as a kind of lethargy, as on that Sunday when he stays home simply out of laziness and admits that he &#8220;was a little bored.&#8221; Thus, even from an absurd perspective, the character retains a peculiar opacity. He is not the Don Juan, nor the Don Quixote of absurdity; indeed, one might even be inclined to believe he is the Sancho Panza. He is there, he exists, and we cannot fully understand or judge him; he lives, and that is the sole literary density that can justify him in our eyes.</p><p>    Yet one must not regard The Stranger as an entirely gratuitous work. Mr. Camus distinguishes&#8212; as we have said&#8212;between the sentiment and the notion of the absurd. On this subject he writes: &#8220;Like great works, profound sentiments always signify more than they are conscious of saying&#8230; Great sentiments carry with them their splendid or miserable universe.&#8221; And he adds a little further on: &#8220;The sentiment of the absurd is not, however, the notion of the absurd. It is its foundation&#8212;that is all. It does not reduce to it&#8230;&#8221; One might say that The Myth of Sisyphus aims to furnish us with that notion and that The Stranger seeks to inspire in us that sentiment. The order in which the two works appeared seems to confirm this hypothesis; The Stranger, published first, plunges us unadorned into the &#8220;climate&#8221; of the absurd; the essay comes later to illuminate the landscape. Now, the absurd is that divorce, that disjunction. The Stranger is therefore a novel of displacement, of divorce, of dislocation. Hence its ingenious construction: on the one hand the daily, amorphous flux of lived reality, and on the other the edifying re-composition of that reality by human reason and discourse. It is essential that the reader, having been first confronted with pure reality, later finds it again, unrecognized, in its rational transposition. From there is born the sentiment of the absurd&#8212;that is, our impotence in thinking, with our concepts, with our words, the events of the world.</p><p>    Meursault buries his mother, takes a mistress, commits a crime. These various facts will be recounted at his trial by grouped witnesses, explained by the public prosecutor: Meursault will have the impression that one is speaking of another. Everything is constructed so as to suddenly bring about the explosion of Marie, who, having testified in court with a narrative composed according to human rules, bursts into tears and exclaims &#8220;that it was not so, that there was something else, that they were forcing her to say the opposite of what she truly thought.&#8221; These maneuvers of ice are commonly employed since Les Faux Monnayeurs. That is not Mr. Camus&#8217;s originality. But the problem he must resolve imposes on him an original form: so that we may feel the gap between the public prosecutor&#8217;s conclusions and the true circumstances of the murder, so that when we close the book we retain the impression of an absurd justice that can never understand or reach the facts it seeks to punish, it is necessary that we have first been put in contact with reality, or with one of those circumstances. Yet to establish that contact Mr. Camus, like the public prosecutor, has only words and concepts at his disposal; he must describe with words, by assembling thoughts, the world before words. The first part of The Stranger might be entitled&#8212;as a recent book might be&#8212;&#8220;Translated from Silence.&#8221; Here we touch on a malady common to many contemporary writers&#8212;the early manifestations of which I see in Jules Renard; I would call it: the haunting of silence. Mr. Paulhan would undoubtedly regard it as an effect of literary terrorism. It has taken a thousand forms&#8212;from the automatic writing of the surrealists to the famous &#8220;theater of silence&#8221; of J.-J. Bernard. It is that silence which, as Heidegger says, is the authentic mode of speech. Only he who is capable of speech remains silent. Mr. Camus talks much&#8212;in The Myth of Sisyphus he even chatters. And yet he confides his love of silence. He quotes Kierkegaard&#8217;s phrase, &#8220;The surest mutism is not to remain silent but to speak,&#8221; and he himself adds that &#8220;a man is more a man by the things he keeps silent about than by the things he says.&#8221; Thus, in The Stranger, he has undertaken to be silent.</p><p>    But how does one remain silent with words? How does one render, with concepts, the unthinkable and disordered succession of presents? This wager involves resorting to a new technique.</p><p>    What is this technique? I was told, &#8220;It is Kafka written by Hemingway.&#8221; I confess that I have not detected Kafka here. Mr. Camus&#8217;s views are entirely terrestrial. Kafka is the novelist of impossible transcendence&#8212;for him, the universe is laden with signs which we do not understand; there is a reverse side to the decor. For Mr. Camus, the human drama is, on the contrary, the absence of any transcendence: &#8220;I do not know whether this world has a meaning that exceeds me. But I know that I do not know that meaning and that, for the moment, it is impossible for me to know it. What would it mean for me to have a meaning beyond my condition? I can understand only in human terms what I touch, what resists me.&#8221;</p><p>    It is not, therefore, a matter of finding word arrangements that might hint at an inhuman and indecipherable order: the inhuman, for him, is simply disorder&#8212;the mechanical. Nothing shady, nothing disturbing, nothing insinuated: The Stranger offers us a succession of luminous views. If they dislocate us, it is solely by their number and by the absence of any link that unites them. Mornings, clear evenings, relentless afternoons&#8212;these are his favorite hours; the perpetual summer of Algiers is his season. Night has little place in his universe. If he speaks of it, he does so thus: &#8220;I awoke with stars on my face. Sounds of the countryside rose to me. Scents of night, of earth and salt refreshed my temples. The marvelous peace of that sleeping summer entered me like a tide.&#8221;</p><p>    The writer of these lines is as far removed as possible from the anxieties of a Kafka; he is at ease in the heart of disorder&#8212;even if the obstinate blindness of nature may annoy him, it reassures him, for its irrationality is only a negative: the absurd man is a humanist, he knows only the goods of this world.</p><p>    The comparison with Hemingway seems more fruitful. The kinship between the two styles is evident. In both texts the sentences are short&#8212;each one refusing to profit from the momentum of the previous sentence; each is like a snapshot of a gesture, of an object. To every new gesture, to every new object corresponds a new sentence. And yet I am not entirely satisfied: the existence of an &#8220;American&#8221; narrative technique has, no doubt, served Mr. Camus. I doubt that it has, strictly speaking, influenced him. Even in Death in the Afternoon&#8212;which is not a novel&#8212;Hemingway retains that jerky mode of narration in which each sentence emerges from nothing by a sort of respiratory spasm: his style is his own. We already know that Mr. Camus possesses another style&#8212;a style of ceremony. Moreover, in The Stranger itself he sometimes raises his tone; then the sentence resumes a broader, continuous flow: &#8220;The cry of the newsvendors in the already relaxed air, the last birds in the square, the call of the sandwich vendors, the lament of the trams on the high curves of the city, and that murmur of the sky before the night tipped over the port&#8212;all this composed for me an itinerary of the blind that I had known well before entering prison.&#8221;</p><p>    Through the gasping narration of Meursault, I glimpse, in transparency, a broader poetic prose underlying his work&#8212;a mode of expression that must be uniquely Mr. Camus&#8217;s. If The Stranger bears such visible traces of the American technique, it is because it is a deliberate borrowing. Mr. Camus has chosen, among the instruments available to him, that which seemed best suited to his purpose. I doubt that he will continue to use it in his future works.</p><p>    Let us now examine more closely the fabric of the narrative so that we may better understand its methods. &#8220;Men also secrete the inhuman,&#8221; writes Mr. Camus. In certain moments of lucidity the mechanical aspect of their gestures&#8212;their pantomime devoid of meaning&#8212;renders all that surrounds them stupid. This is what must first be conveyed: The Stranger must immediately plunge us into a state of discomfort before the inhumanity of man. But what are those singular occasions that can provoke in us such discomfort? The Myth of Sisyphus provides an example: &#8220;A man speaks on the telephone, behind a glass partition; one does not hear him, but one sees his expression without resonance: one wonders why he lives.&#8221;</p><p>    Here we are informed&#8212;almost too much so, for the example reveals a certain bias on the part of the author. Indeed, the gesture of the man on the phone&#8212;whom you do not hear&#8212;is only relatively absurd: it belongs to a truncated circuit. Open the door, put your ear to the receiver: the circuit is restored, human activity resumes its meaning. One might then, if being entirely honest, say that there are only relative absurdities&#8212;and only by reference to &#8220;absolute rationals.&#8221; But this is not a question of honesty; it is a matter of art. Mr. Camus&#8217;s method is altogether ingenious: between the characters he describes and the reader he interposes a glass partition. What is more inane than men behind a pane? It seems that it lets everything pass, stopping only one thing&#8212;the meaning of their gestures. What remains is to choose the glass: it will be the consciousness of the Stranger. Indeed, it is a transparency: we see everything it sees. Only it has been constructed so as to be transparent to things and opaque to meanings.</p><p>    &#8220;From that moment everything sped up. Men advanced toward the beer with a sheet. The priest, his followers, the director, and I went out. In front of the door there was a lady whom I did not know: &#8216;Mr. Meursault,&#8217; said the director. I did not catch the lady&#8217;s name; I only understood that she was the designated nurse. She inclined her bony face without a smile, and then we lined up to let the body pass.&#8221;</p><p>    Men dance behind a glass. Between them and the reader a consciousness&#8212;almost nothing, a pure translucence, a sheer passivity that registers all events&#8212;has been interposed. And the trick is complete: precisely because it is passive, consciousness registers only the facts. The reader does not notice this interposition. But then what is the postulate implied by this type of narrative? In short, what was once a melodious organization has been rendered an addition of invariant elements; it is claimed that the succession of movements is rigorously identical to the act taken as a whole. Are we not dealing here with the analytic postulate, which asserts that all reality is reducible to a sum of elements? Now, if analysis is the instrument of science, it is also the instrument of humor. If I wish to describe a rugby match and write, &#8220;I saw grown men in short trousers fighting and tumbling on the ground to pass a leather ball between two wooden posts,&#8221; I have summed up what I saw; yet I have deliberately omitted its meaning&#8212;I have made humor.</p><p>    Mr. Camus&#8217;s narrative is analytic and humorous. It lies&#8212;as every artist does&#8212;by claiming to reproduce naked experience while slyly filtering out all the signifying links, which are also part of experience. This is what long ago Hume did when he declared that in experience one discovers only isolated impressions. It is what neo-realist Americans continue to do today when they deny that there is between phenomena anything other than external relations. Against them, contemporary philosophy has shown that meanings, too, are immediate data. But that would lead us too far. Let it suffice to say that the universe of the absurd man is the analytic world of the neo-realists. Literarily, the procedure has proven itself: it is that of L&#8217;Ing&#233;nu or of Microm&#233;gas; it is that of Gulliver. For the seventeenth century also had its strangers&#8212;generally &#8220;noble savages&#8221; who, transported into an unknown civilization, perceived facts before grasping their meaning. Was not the effect of this dislocation precisely to provoke in the reader the sentiment of the absurd? Mr. Camus seems to recall this on several occasions, particularly when he shows his hero reflecting on the reasons for his imprisonment.</p><p>    Now it is this analytic procedure that explains the use in The Stranger of the American technique. The presence of death at the end of our road has reduced our future to smoke; our life is &#8220;without a tomorrow,&#8221; it is a succession of presents. What does this mean but that the absurd man applies his analytic mind to time? Where Bergson saw an indivisible organization, his eye sees only a series of instants. It is the plurality of incommunicable instants that will finally account for the plurality of beings. What our author borrows from Hemingway is thus the discontinuity of his choppy sentences, which mirrors the discontinuity of time. We come then to better understand the cutting of his narrative: each sentence is a present. But not an indefinite present that blurs into and extends the present that follows. The sentence is crisp, without smudges, self-contained; it is separated from the next sentence by a void, just as Descartes&#8217;s instant is separated from the next. Between each sentence and the following one the world annihilates itself and is reborn: language, as soon as it is uttered, is a creation ex nihilo; a sentence in The Stranger is an island. And we cascade from sentence to sentence, from void to void.</p><p>    It is to accentuate the solitude of each phrasal unit that Mr. Camus chose to compose his narrative in the perfect past tense. The defined past is the time of continuity: &#8220;He strolled for a long time&#8221; calls to mind the pluperfect, a future; the reality of the sentence is the verb&#8212;it is the act, with its transitive character, with its transcendence. &#8220;He has strolled for a long time&#8221; conceals the verbality of the verb; the verb is split in two: on one side lies a past participle that has lost all transcendence, inert as a thing, and on the other the verb &#8220;to be,&#8221; which carries only the sense of a copula, joining the participle to the noun as an attribute to the subject; the transitive character of the verb has evaporated, the sentence has frozen; its reality now is the noun. Instead of acting as a bridge between the past and the future, it is nothing but a small, isolated substance that is self-sufficient. Moreover, if one takes care to reduce it as much as possible to the main clause, its internal structure becomes of perfect simplicity&#8212;and thus gains in cohesion. It is truly an inseparable, an atom of time. Naturally one does not organize sentences among themselves: they are purely juxtaposed; in particular one avoids any causal links, which would introduce into the narrative an embryo of explanation and impose on the pure succession of instants an order other than mere succession. One writes: &#8220;A moment later, she asked me if I loved her. I replied that it meant nothing, but that it seemed to me that no. She looked sad. But while preparing lunch, and about nothing in particular, she laughed once again in such a way that I kissed her. It was at that moment that the sounds of an argument broke out at Raymond&#8217;s.&#8221;</p><p>    We emphasize two sentences that most carefully conceal any causal link under the pure appearance of succession. When it is absolutely necessary to allude in one sentence to the previous one, words such as &#8220;and,&#8221; &#8220;but,&#8221; &#8220;then,&#8221; &#8220;it was at that moment that&#8230;&#8221; are used&#8212;which evoke nothing except pure disjunction, opposition, or mere addition. The relations between these temporal units are external, like those that neo-realism establishes among things; reality appears without being ushered in and disappears without being destroyed&#8212;the world collapses and is reborn with each pulse of time. But let us not imagine that it produces itself: it is inert. Any activity on its part would tend to substitute formidable powers for the reassuring disorder of chance. A nineteenth-century naturalist would have written: &#8220;A bridge spanned the river.&#8221; Mr. Camus refuses anthropomorphism. He would say: &#8220;Above the river, there was a bridge.&#8221; Thus the thing immediately presents us with its passivity. It is there, simply, undifferentiated: &#8220;&#8230; There were four black men in the room&#8230; in front of the door there was a lady whom I did not know&#8230; In front of the door there was the car&#8230; Next to it, there was the paymaster&#8230;&#8221; It was said of Renard that he would eventually write: &#8220;The hen lays.&#8221; Mr. Camus and many contemporary authors would write: &#8220;There is the hen, and it lays.&#8221; They love things for themselves; they do not wish to dilute them in the flux of duration. &#8220;There is water&#8221;: here is a little piece of eternity, passive, impenetrable, incommunicable, glittering&#8212;what sensual delight if one can touch it! For the absurd man, it is the sole good in this world. That is why the novelist prefers to an organized narrative that scintillation of little flashes without a tomorrow, each one a moment of voluptuous pleasure; that is why Mr. Camus, in writing The Stranger, may believe that he is silent: his sentence does not belong to the universe of discourse, it has neither ramifications, nor extensions, nor internal structure; it could be defined, like Val&#233;ry&#8217;s Sylph:</p><div class="pullquote"><p><em><strong>Unseen and unknown:<br>For the time of a bare breast<br>Between two shirts.</strong></em></p></div><p>    It is measured very precisely by the time of a silent intuition. Under these conditions, can we speak of a whole that is Mr. Camus&#8217;s novel?</p><p>    Every sentence in his book is equivalent, just as every experience of the absurd man is; each stands on its own and consigns the others to nothingness; yet, except in the rare moments when the author, unfaithful to his principle, indulges in poetry, none stands out against the others. Even the dialogues are integrated into the narrative: dialogue is the moment of explanation, of signification; to give it a privileged place would be to admit that meanings exist. Mr. Camus reduces it, summarizes it, often expresses it in indirect style, denying it any typographical privilege, so that the spoken sentences appear as events akin to the others&#8212;glittering for an instant and then disappearing like a flash of heat, like a sound, like a scent. Thus, when one begins to read the book, it does not at first seem that one is in the presence of a novel but rather a monotonous melody&#8212;the nasal song of an Arab. One might then think that the book will resemble one of those airs Courteline speaks of, which &#8220;go away and never return&#8221; and then stop abruptly without any apparent reason. But gradually the work organizes itself before the reader&#8217;s eyes; it reveals the solid substructure that supports it. Not a single detail is superfluous, not one that is not later resumed and poured into the debate; and, with the book closed, we understand that it could not have begun otherwise, that it could not have another ending: in this world, presented to us as absurd and from which causality has been meticulously extirpated, not the slightest incident is without weight&#8212;each contributes to leading the hero toward the crime and toward capital punishment.</p><p>    The Stranger is a classic work&#8212;a work of order composed regarding the absurd and against the absurd. Is that entirely what the author intended? I do not know; it is my opinion as a reader.</p><p>    And how shall we classify this work, so dry and crisp, though composed in its apparent disorder, so &#8220;human,&#8221; so unswervingly transparent once one holds its key? We could scarcely call it a narrative&#8212;the narrative explains and coordinates even as it recounts, substituting causal order for mere chronological succession. Mr. Camus calls it a &#8220;novel.&#8221; And yet a novel demands continuous duration, a becoming, the manifest presence of time&#8217;s irreversibility. It is not without hesitation that I would attribute that name to this succession of inert presents which, beneath, reveals the mechanical economy of a constructed edifice. Or else it would be, in the manner of Zadig and Candide, a short novel by a moralist&#8212;with a discreet note of satire and ironic portraits&#8212;which, despite the influence of German existentialists and American novelists, remains very much in essence akin to a Voltaire tale.</p><p>February 1943, Jean-Paul Sartre</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Calvin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p></p>]]></content:encoded></item><item><title><![CDATA[The rational take on Founder Mode]]></title><description><![CDATA[A less toxic Founder Mode requires self-awareness]]></description><link>https://blog.cal.vin/p/the-rational-take-on-founder-mode</link><guid isPermaLink="false">https://blog.cal.vin/p/the-rational-take-on-founder-mode</guid><dc:creator><![CDATA[Calvin Li]]></dc:creator><pubDate>Thu, 26 Sep 2024 14:48:52 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!N2l5!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!N2l5!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!N2l5!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg 424w, https://substackcdn.com/image/fetch/$s_!N2l5!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg 848w, https://substackcdn.com/image/fetch/$s_!N2l5!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!N2l5!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!N2l5!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg" width="1280" height="720" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/e7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:720,&quot;width&quot;:1280,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Paul Graham Audio Essays - Founder mode&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Paul Graham Audio Essays - Founder mode" title="Paul Graham Audio Essays - Founder mode" srcset="https://substackcdn.com/image/fetch/$s_!N2l5!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg 424w, https://substackcdn.com/image/fetch/$s_!N2l5!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg 848w, https://substackcdn.com/image/fetch/$s_!N2l5!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!N2l5!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe7fb2b34-0803-47a7-a5a6-d86af6f32fa1_1280x720.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p><a href="https://paulgraham.com/foundermode.html">Paul Graham&#8217;s essay on Founder Mode</a> highlights a perspective that addresses a challenge that exists across all companies at all scales. Here&#8217;s the rational take:</p><ol><li><p>Hiring and delegation are hard, understandably.</p></li><li><p>If neither or both are not done adequately, then micromanaging is a reasonable, however unfortunate, strategy for compensating.</p></li></ol><p>This framing critiques the platonic ideals of business school dogma that micromanaging is strictly bad without factoring in the challenges of hiring and delegation. From this viewpoint, micromanagement can often be a necessary evil to maintain effectiveness. It should not the preferred avenue.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Calvin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>Perhaps it&#8217;s called &#8220;Founder Mode&#8221; because founders require a broader skillset than professional managers. As a result, many founders have less-developed hiring and delegation skills, especially in non-core areas. This leads them to need to micromanage to compensate for their limitations. A healthier, less toxic, discourse around Founder Mode requires acknowledging that micromanagement often arises from a leader&#8217;s skill gaps&#8212;not from a blanket justification for making direct reports miserable.</p><h1>Founder Mode in early-stage startups</h1><p>I disagree with the following Y Combinator video that Founder mode only applies to late-stage startups. The challenges of hiring and delegation are pervasive, whether you&#8217;re making your first hire or your 1000th with the complexities of a large organization with nested reporting structures.</p><div id="youtube2-FlonCU9txXI" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;FlonCU9txXI&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/FlonCU9txXI?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h2>Example from my own company</h2><p>As someone with an engineering background, I&#8217;m confident in my ability to hire and delegate engineering work. But a successful startup requires much more than just engineering; there&#8217;s product development, design, sales, marketing, operations, HR, accounting, and more.</p><p>At large tech companies, I&#8217;ve been exposed to most of these functions&#8212;except accounting. In my early-stage, cash-constrained startup, accounting is critical, yet I don&#8217;t know what makes a good accountant or how to delegate those tasks effectively. This gap in my knowledge means I end up in &#8220;Founder Mode&#8221; when it comes to accounting&#8212;an area where I lack confidence and, unfortunately, overcompensate by micromanaging.</p><p>There is no such thing as a perfect founder. Life&#8212;and startups&#8212;are full of constraints, and hiring and delegating effectively across all functions is a rare skill. Most of us struggle with it at some point, and it&#8217;s important to acknowledge this rather than perpetuate an idealized, unrealistic image of startup leadership.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Calvin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[The Hydra Game Solved]]></title><description><![CDATA[After solving for Hydra(5), here&#8217;s the general solution to the hydra game]]></description><link>https://blog.cal.vin/p/the-hydra-game-solved</link><guid isPermaLink="false">https://blog.cal.vin/p/the-hydra-game-solved</guid><dc:creator><![CDATA[Calvin Li]]></dc:creator><pubDate>Wed, 01 May 2024 19:44:10 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!xpcR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>This is the prior post with the answer to <em><strong>hydra(5)</strong></em>. In this post, we&#8217;ll be providing the exact sequence for the general hydra problem.</p><div class="digest-post-embed" data-attrs="{&quot;nodeId&quot;:&quot;c2bfc09a-552a-486d-afa6-c86d38ce1b48&quot;,&quot;caption&quot;:&quot;The Hydra Game The game is described in the introduction in the wikipedia page: Hydra Game The Numberphile video (where I learned of the problem) does a great job describing the game (video starts when they describe the problem). If text is more convenient:&quot;,&quot;cta&quot;:null,&quot;showBylines&quot;:true,&quot;size&quot;:&quot;sm&quot;,&quot;isEditorNode&quot;:true,&quot;title&quot;:&quot;hydra(5)&quot;,&quot;publishedBylines&quot;:[{&quot;id&quot;:6452692,&quot;name&quot;:&quot;Calvin Li&quot;,&quot;bio&quot;:&quot;Welcome to my blog! I strive to write at the intersection of philosophy, economics, AI, and technology, and keep it interesting with novel insights into the world we live in.&quot;,&quot;photo_url&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/b5ac185f-a363-4144-9bb9-fc5b1511b0e7_144x144.png&quot;,&quot;is_guest&quot;:false,&quot;bestseller_tier&quot;:null}],&quot;post_date&quot;:&quot;2024-04-26T07:19:00.235Z&quot;,&quot;cover_image&quot;:&quot;https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp&quot;,&quot;cover_image_alt&quot;:null,&quot;canonical_url&quot;:&quot;https://li.cal.vin/p/hydra5&quot;,&quot;section_name&quot;:null,&quot;video_upload_id&quot;:null,&quot;id&quot;:143858982,&quot;type&quot;:&quot;newsletter&quot;,&quot;reaction_count&quot;:0,&quot;comment_count&quot;:0,&quot;publication_id&quot;:null,&quot;publication_name&quot;:&quot;Calvin&quot;,&quot;publication_logo_url&quot;:&quot;https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb5ac185f-a363-4144-9bb9-fc5b1511b0e7_144x144.png&quot;,&quot;belowTheFold&quot;:false,&quot;youtube_url&quot;:null,&quot;show_links&quot;:null,&quot;feed_url&quot;:null}"></div><p>Also wikipedia page: <a href="https://en.wikipedia.org/wiki/Hydra_game">https://en.wikipedia.org/wiki/Hydra_game</a></p><p>See the previous post for an explanation of how the game is played, and the notation I&#8217;m using:</p><p>(<em><strong>[a, b, c, d, e, &#8230;, n], S)</strong></em></p><p>Feel free to scroll down to the solution if the proof isn&#8217;t of interest.</p><h1>Iterating Forward</h1><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!xpcR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!xpcR!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp 424w, https://substackcdn.com/image/fetch/$s_!xpcR!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp 848w, https://substackcdn.com/image/fetch/$s_!xpcR!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp 1272w, https://substackcdn.com/image/fetch/$s_!xpcR!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!xpcR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp" width="1024" height="1024" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1024,&quot;width&quot;:1024,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;A detailed illustration of the 'Infinite Hydra Game' featuring a Hydra that forms an infinity symbol, set against a mythological background. The Hydra, with its numerous heads and tails, loops gracefully to create the infinity shape. The scene is enriched with elements of ancient Greek mythology, including marble columns, a Pantheon-like temple in the distance, and ethereal clouds. Mythological figures, perhaps gods and heroes, are depicted as small silhouettes, engaged in a battle with the Hydra using spears and swords. The setting conveys a timeless battle of epic proportions, surrounded by an air of legend and mystique.&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="A detailed illustration of the 'Infinite Hydra Game' featuring a Hydra that forms an infinity symbol, set against a mythological background. The Hydra, with its numerous heads and tails, loops gracefully to create the infinity shape. The scene is enriched with elements of ancient Greek mythology, including marble columns, a Pantheon-like temple in the distance, and ethereal clouds. Mythological figures, perhaps gods and heroes, are depicted as small silhouettes, engaged in a battle with the Hydra using spears and swords. The setting conveys a timeless battle of epic proportions, surrounded by an air of legend and mystique." title="A detailed illustration of the 'Infinite Hydra Game' featuring a Hydra that forms an infinity symbol, set against a mythological background. The Hydra, with its numerous heads and tails, loops gracefully to create the infinity shape. The scene is enriched with elements of ancient Greek mythology, including marble columns, a Pantheon-like temple in the distance, and ethereal clouds. Mythological figures, perhaps gods and heroes, are depicted as small silhouettes, engaged in a battle with the Hydra using spears and swords. The setting conveys a timeless battle of epic proportions, surrounded by an air of legend and mystique." srcset="https://substackcdn.com/image/fetch/$s_!xpcR!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp 424w, https://substackcdn.com/image/fetch/$s_!xpcR!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp 848w, https://substackcdn.com/image/fetch/$s_!xpcR!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp 1272w, https://substackcdn.com/image/fetch/$s_!xpcR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><h2>Goal: Reduce ([1, 1, &#8230;, 1, X], S) &#8594; ([1, 1, &#8230;, 1, 1], ??)</h2><h3>Base Case: ([1], S)</h3><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(a)\\ ([1], S) \\rightarrow S;\\ \\text{on the S-th step, the last head is chopped}&quot;,&quot;id&quot;:&quot;KPVVIYZROF&quot;}" data-component-name="LatexBlockToDOM"></div><h3>1st index: ([1 + a, &#8230;], S) &#8594; ([1, &#8230;], ??)</h3><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(b)\\ ([1 + a, \\ldots], S) \\rightarrow ([1, \\ldots], S + a)&quot;,&quot;id&quot;:&quot;GCMBWSKHTO&quot;}" data-component-name="LatexBlockToDOM"></div><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Calvin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h3>2nd index: ([1, 1 + b, &#8230;], S) &#8594; ([1, 1, &#8230;], ??)</h3><p>Starting with <em><strong>b = 1</strong></em>:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(c)\\ ([1, 1+1, \\ldots], S) \\rightarrow ([1 + S, 1, \\ldots], S + 1) \\to ([1, 1, \\ldots], 2 \\cdot (S + 1) - 1)&quot;,&quot;id&quot;:&quot;GRXTXOATOH&quot;}" data-component-name="LatexBlockToDOM"></div><p>Let&#8217;s define the function that decrements the 2nd index by 1 as <em><strong>B(x)</strong></em><strong>:</strong></p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;B(x) = 2\\cdot(S+1) - 1&quot;,&quot;id&quot;:&quot;BZYGASQZZN&quot;}" data-component-name="LatexBlockToDOM"></div><p>And generally define the n-th nesting of any function to be:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;F^n(x) = \\underbrace{F(F(\\cdots F}_{n\\, \\text{times}}(x)\\cdots))\n&quot;,&quot;id&quot;:&quot;XERMYRGWNV&quot;}" data-component-name="LatexBlockToDOM"></div><p>Continuing with <em><strong>B = 2</strong></em>:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(d)\\ ([1, 1+2, \\ldots], S) \\rightarrow ([1 + S, 1 + 1, \\ldots], S + 1) \\to ([1, 1 + 1, \\ldots], 2 \\cdot (S + 1) - 1) &quot;,&quot;id&quot;:&quot;VQSOJIZPIT&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(d, cont) \\to ([1, 1, \\ldots], 2 \\cdot (2 \\cdot(S + 1) - 1 + 1) - 1) \\to ([1, 1, \\ldots], 4 \\cdot (S + 1) - 1)&quot;,&quot;id&quot;:&quot;AHFCBFJJNY&quot;}" data-component-name="LatexBlockToDOM"></div><p>Noticing the same trend as <em><strong>b = 1</strong></em>, generalizing the function B:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(e)\\ ([1, 1+b, \\ldots], S) \\to ([1, 1, \\ldots], 2^b \\cdot (S + 1) - 1)&quot;,&quot;id&quot;:&quot;VBWFJTOJVK&quot;}" data-component-name="LatexBlockToDOM"></div><p>and if &#8230; is all 0&#8217;s, then:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(f)\\ ([1, b], S) \\to ([1], 2^{b} \\cdot (S + 1) - 1) \\to 2^{b}\\cdot(S+1) - 1, \\text{per (a) &amp; (e)}&quot;,&quot;id&quot;:&quot;ZNJIAGWCYM&quot;}" data-component-name="LatexBlockToDOM"></div><p>Quick sanity check, <em><strong>hydra(2)</strong></em> = 3, which this formula conforms to.</p><p>Generally:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;2^{b}\\cdot(S+1) - 1 = B^b(S)&quot;,&quot;id&quot;:&quot;CEFUJOTDCG&quot;}" data-component-name="LatexBlockToDOM"></div><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Calvin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h3>3rd index: ([1, 1, 1 + c, &#8230;], S) &#8594; ([1, 1, 1, &#8230;], ??)</h3><p>Starting with <em><strong>c = 1</strong></em>:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(g)\\ ([1, 1, 1+1, \\ldots], S) \\rightarrow ([1, 1 + S, 1, \\ldots], S + 1) \\to ([1, 1, 1, \\ldots], 2^S \\cdot (S + 2) - 1 = B^S(S + 1))&quot;,&quot;id&quot;:&quot;YQPRIEKMNT&quot;}" data-component-name="LatexBlockToDOM"></div><p>Let&#8217;s define a function <em><strong>C(x)</strong></em>:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;C(x) = B^x(x+1) = 2^S\\cdot(S+2) - 1&quot;,&quot;id&quot;:&quot;DQHPXDPPXM&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(g, rewritten)\\ ([1, 1, 1+1, \\ldots], S) \\to C(S)&quot;,&quot;id&quot;:&quot;NADTEXLWJH&quot;}" data-component-name="LatexBlockToDOM"></div><p>with <em><strong>c = 2</strong></em>:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(h)\\ ([1, 1, 1+2, \\ldots], S) \\to ([1, 1 + S, 1 + 1, \\ldots], S + 1) \\to ([1, 1, 1 + 1, \\ldots], C(S)) \\to ([1, 1, 1, \\ldots], C^2(S))&quot;,&quot;id&quot;:&quot;TPOGSJQLHH&quot;}" data-component-name="LatexBlockToDOM"></div><p>generalizing&#8230;</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(i)\\ ([1, 1, 1+c, \\ldots], S) \\to ([1, 1 + S, 1 + c - 1, \\ldots], S + 1) \\to ([1, 1, 1 + c - 1, \\ldots], C(S)) \\to ([1, 1, 1, \\ldots], C^c(S))&quot;,&quot;id&quot;:&quot;LWHOCRUZEY&quot;}" data-component-name="LatexBlockToDOM"></div><p>if &#8230; were 0&#8217;s:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(j)\\ ([1, 1, c], S) \\to ([1, 1 + S, c - 1], S + 1) \\to ([1, 1, c - 1], C(S)) \\to ([1, 1, 0], C^c(S))&quot;,&quot;id&quot;:&quot;ZEYXLIIZZB&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(j, cont)  \\to ([1, 0, 0], B(C^c(S))) \\to B(C^c(S))&quot;,&quot;id&quot;:&quot;RDAXVGELXW&quot;}" data-component-name="LatexBlockToDOM"></div><p>Quick sanity check for <em><strong>hydra(3)</strong></em> = 11:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;([1, 1, 1], 1) \\to B(C(1)) = 2 \\cdot (C(1) + 1) - 1 = 2 \\cdot (2^1 \\cdot (1 + 2) - 1) + 1) - 1 = 11&quot;,&quot;id&quot;:&quot;OPCAZNTMSZ&quot;}" data-component-name="LatexBlockToDOM"></div><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Calvin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h3>4th index: ([1, 1, 1, 1 + d, &#8230;], S) &#8594; ([1, 1, 1, 1, &#8230;], ??)</h3><p>Starting with <em><strong>d = 1</strong></em>:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(k)\\ ([1, 1, 1, 1+1, \\ldots], S) \\to ([1, 1, 1 + S, 1, \\ldots], S + 1) \\to ([1, 1, 1, 1, \\ldots], C^S(S+1))&quot;,&quot;id&quot;:&quot;XWFPMRWZDJ&quot;}" data-component-name="LatexBlockToDOM"></div><p>Let&#8217;s define a function <em><strong>D(x)</strong></em>:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;D(x) = C^x(x+1)&quot;,&quot;id&quot;:&quot;KGIDJQGRSN&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(k, rewritten)\\ ([1, 1, 1, 1+1, \\ldots], S) \\to D(S)&quot;,&quot;id&quot;:&quot;PMZHPHSZDN&quot;}" data-component-name="LatexBlockToDOM"></div><p>with <em><strong>d = 2</strong></em>:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(l)\\ ([1, 1, 1, 1+2, \\ldots], S) \\to ([1, 1, 1 + S, 1 + 1, \\ldots], S + 1) \\to ([1, 1, 1, 1 + 1, \\ldots], D(S)) \\to ([1, 1, 1, \\ldots], D^2(S))&quot;,&quot;id&quot;:&quot;ZZYMZNPXJF&quot;}" data-component-name="LatexBlockToDOM"></div><p>generalizing&#8230;</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(m)\\ ([1, 1, 1, 1+d, \\ldots], S) \\to ([1, 1, 1 + S, 1 + d - 1, \\ldots], S + 1) \\to ([1, 1, 1, 1 + d - 1, \\ldots], C(S)) \\to ([1, 1, 1, \\ldots], D^d(S))&quot;,&quot;id&quot;:&quot;XJYZDMKUOT&quot;}" data-component-name="LatexBlockToDOM"></div><p>if &#8230; were 0&#8217;s:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(n)\\ ([1, 1, 1, d], S) \\to ([1, 1, 1 + S, d - 1], S + 1) \\to ([1, 1, d - 1], D(S)) \\to ([1, 1, 1, 0], D^d(S))&quot;,&quot;id&quot;:&quot;ILNHWWXSAX&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(n, cont)  \\to ([1, 0, 0, 0], B(C(D^d(S)))) \\to B(C(D^d(S)))&quot;,&quot;id&quot;:&quot;GWTXPPSBUX&quot;}" data-component-name="LatexBlockToDOM"></div><p>Quick sanity check for <em><strong>hydra(4)</strong></em> = 1114111:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;B(C(D(1))) = B(C(C^1(1 + 1))) = B(C(B^2(2 + 1))) = B(C(2^2\\cdot (3 + 1) - 1))&quot;,&quot;id&quot;:&quot;NBCKQZXLLU&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot; = B(C(15)) = B(2^{15} * (15 + 2) - 1) = B(557055) = 2 \\cdot ( 557055 + 1) - 1 = 1114111&quot;,&quot;id&quot;:&quot;XXFELIEKYH&quot;}" data-component-name="LatexBlockToDOM"></div><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Calvin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h1>The Solution</h1><p>Let&#8217;s define a function that describes the number of cuts required to decrement the <em>i</em>-th depth of the hydra while cleaning up the subsequent spawned heads:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;([1, 1, 1, \\ldots, h_i, \\ldots], x) \\to ([1, 1, 1, \\ldots, h_i - 1, ...], F_i(x))&quot;,&quot;id&quot;:&quot;SXSLWSNKDE&quot;}" data-component-name="LatexBlockToDOM"></div><p>Then, based on what&#8217;s been demonstrated in the iterated section:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;F_{i+1}(x) = F_i^x(x + 1)&quot;,&quot;id&quot;:&quot;GJPVMPSIGZ&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;where\\ F_1(x) = 2\\cdot(x + 1) - 1 = 2x + 1&quot;,&quot;id&quot;:&quot;MZIKVIHPHG&quot;}" data-component-name="LatexBlockToDOM"></div><p>And the answer to the linear hydra game of depth <em><strong>n</strong></em> is:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;F_1(F_2(F_3(\\ldots F_{n-1}(F_n(1))\\ldots)))&quot;,&quot;id&quot;:&quot;CPNDFQSABL&quot;}" data-component-name="LatexBlockToDOM"></div><p>To figure out any arbitrary hydra configuration, <em><strong>([a, b, c, d, e, f, &#8230; ], S)</strong></em> you nest the solution for each index inversely with S in the middle to reduce the problem to ([1, 1, 1, &#8230;, X], ??). Where ?? is the result of applying all necessary F starting from the first index to reduce the hydra to a single stem.</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/the-hydra-game-solved?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Thank you for reading Calvin. This post is public so feel free to share it.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/the-hydra-game-solved?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://blog.cal.vin/p/the-hydra-game-solved?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><p></p>]]></content:encoded></item><item><title><![CDATA[hydra(5)]]></title><description><![CDATA[Exact answer to the hydra game's 5th value. First 4 values are 1, 3, 11, 1114111]]></description><link>https://blog.cal.vin/p/hydra5</link><guid isPermaLink="false">https://blog.cal.vin/p/hydra5</guid><dc:creator><![CDATA[Calvin Li]]></dc:creator><pubDate>Fri, 26 Apr 2024 07:19:00 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!Xwts!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h1>The Hydra Game</h1><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Xwts!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Xwts!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp 424w, https://substackcdn.com/image/fetch/$s_!Xwts!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp 848w, https://substackcdn.com/image/fetch/$s_!Xwts!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp 1272w, https://substackcdn.com/image/fetch/$s_!Xwts!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Xwts!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp" width="540" height="540" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:false,&quot;imageSize&quot;:&quot;normal&quot;,&quot;height&quot;:1024,&quot;width&quot;:1024,&quot;resizeWidth&quot;:540,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;An imaginative depiction of the Hydra games, inspired by the Hercules mythology. The scene shows a large, multi-headed Hydra looming menacingly over a coliseum filled with spectators. Each head of the Hydra has a distinct, terrifying expression, and its serpentine body twists dramatically around the ancient stone architecture. In the foreground, a heroic figure resembling Hercules, wearing traditional Greek armor, stands poised with a sword, ready to battle the creature. The atmosphere is tense and dramatic, with dark clouds gathering above and the crowd visibly anxious yet captivated.&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="An imaginative depiction of the Hydra games, inspired by the Hercules mythology. The scene shows a large, multi-headed Hydra looming menacingly over a coliseum filled with spectators. Each head of the Hydra has a distinct, terrifying expression, and its serpentine body twists dramatically around the ancient stone architecture. In the foreground, a heroic figure resembling Hercules, wearing traditional Greek armor, stands poised with a sword, ready to battle the creature. The atmosphere is tense and dramatic, with dark clouds gathering above and the crowd visibly anxious yet captivated." title="An imaginative depiction of the Hydra games, inspired by the Hercules mythology. The scene shows a large, multi-headed Hydra looming menacingly over a coliseum filled with spectators. Each head of the Hydra has a distinct, terrifying expression, and its serpentine body twists dramatically around the ancient stone architecture. In the foreground, a heroic figure resembling Hercules, wearing traditional Greek armor, stands poised with a sword, ready to battle the creature. The atmosphere is tense and dramatic, with dark clouds gathering above and the crowd visibly anxious yet captivated." srcset="https://substackcdn.com/image/fetch/$s_!Xwts!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp 424w, https://substackcdn.com/image/fetch/$s_!Xwts!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp 848w, https://substackcdn.com/image/fetch/$s_!Xwts!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp 1272w, https://substackcdn.com/image/fetch/$s_!Xwts!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4c21782-205d-4c30-8073-19ba68c4fae5_1024x1024.webp 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">of course this is GenAI</figcaption></figure></div><p>The game is described in the introduction in the wikipedia page: <a href="https://en.wikipedia.org/wiki/Hydra_game">Hydra Game</a> </p><p>The Numberphile video (where I learned of the problem) does a great job describing the game (video starts when they describe the problem). If text is more convenient:</p><blockquote><p>Start with a linear tree of length N. You may only chop leaves in this tree. When you chop a leaf of the tree that&#8217;s not a child of the root, you add new siblings to its parent. The number of new siblings is equal to the number of steps taken so far. So the first chop adds 1 sibling to the parent, the second chop adds 2 siblings, etc.</p><p>In this particular variant, you always choose the &#8220;right-most&#8221; leaf, which is equivalent to saying you always choose leaves closest to the root, and the leaves on a node are essentially indistinguishable. The drawing below demonstrates the hydra game sequence of 3.</p></blockquote><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!vgQh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F033027ae-85c4-4f0d-80ed-bfcc9d5e7bf2_2880x419.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!vgQh!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F033027ae-85c4-4f0d-80ed-bfcc9d5e7bf2_2880x419.png 424w, https://substackcdn.com/image/fetch/$s_!vgQh!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F033027ae-85c4-4f0d-80ed-bfcc9d5e7bf2_2880x419.png 848w, https://substackcdn.com/image/fetch/$s_!vgQh!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F033027ae-85c4-4f0d-80ed-bfcc9d5e7bf2_2880x419.png 1272w, https://substackcdn.com/image/fetch/$s_!vgQh!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F033027ae-85c4-4f0d-80ed-bfcc9d5e7bf2_2880x419.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!vgQh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F033027ae-85c4-4f0d-80ed-bfcc9d5e7bf2_2880x419.png" width="1456" height="212" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/033027ae-85c4-4f0d-80ed-bfcc9d5e7bf2_2880x419.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:212,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;undefined&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="undefined" title="undefined" srcset="https://substackcdn.com/image/fetch/$s_!vgQh!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F033027ae-85c4-4f0d-80ed-bfcc9d5e7bf2_2880x419.png 424w, https://substackcdn.com/image/fetch/$s_!vgQh!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F033027ae-85c4-4f0d-80ed-bfcc9d5e7bf2_2880x419.png 848w, https://substackcdn.com/image/fetch/$s_!vgQh!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F033027ae-85c4-4f0d-80ed-bfcc9d5e7bf2_2880x419.png 1272w, https://substackcdn.com/image/fetch/$s_!vgQh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F033027ae-85c4-4f0d-80ed-bfcc9d5e7bf2_2880x419.png 1456w" sizes="100vw"></picture><div></div></div></a><figcaption class="image-caption">from wikipedia: <a href="https://en.wikipedia.org/wiki/Hydra_game">https://en.wikipedia.org/wiki/Hydra_game</a></figcaption></figure></div><div id="youtube2-prURA1i8Qj4" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;prURA1i8Qj4&quot;,&quot;startTime&quot;:&quot;957&quot;,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/prURA1i8Qj4?start=957&amp;rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><p>Note <em><strong>hydra(4) = 983,038</strong></em> is likely from wikipedia, and it&#8217;s <strong>incorrect</strong>.</p><h1>tl;dr: the answer</h1><p><em><strong>Update: I&#8217;ve found the general answer here:</strong></em></p><div class="digest-post-embed" data-attrs="{&quot;nodeId&quot;:&quot;2d756c87-c399-4358-b90b-c8f72bee8f61&quot;,&quot;caption&quot;:&quot;This is the prior post with the answer to hydra(5). In this post, we&#8217;ll be providing the exact sequence for the general hydra problem. Also wikipedia page: https://en.wikipedia.org/wiki/Hydra_game See the previous post for an explanation of how the game is played, and the notation I&#8217;m using:&quot;,&quot;cta&quot;:null,&quot;showBylines&quot;:true,&quot;size&quot;:&quot;md&quot;,&quot;isEditorNode&quot;:true,&quot;title&quot;:&quot;The Hydra Game Solved&quot;,&quot;publishedBylines&quot;:[{&quot;id&quot;:6452692,&quot;name&quot;:&quot;Calvin Li&quot;,&quot;bio&quot;:&quot;Welcome to my blog! I strive to write at the intersection of philosophy, economics, AI, and technology, and keep it interesting with novel insights into the world we live in.&quot;,&quot;photo_url&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/b5ac185f-a363-4144-9bb9-fc5b1511b0e7_144x144.png&quot;,&quot;is_guest&quot;:false,&quot;bestseller_tier&quot;:null}],&quot;post_date&quot;:&quot;2024-05-01T19:44:10.667Z&quot;,&quot;cover_image&quot;:&quot;https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f724cb-c266-4ebd-ae47-53067b65ab92_1024x1024.webp&quot;,&quot;cover_image_alt&quot;:null,&quot;canonical_url&quot;:&quot;https://li.cal.vin/p/the-hydra-game-solved&quot;,&quot;section_name&quot;:null,&quot;video_upload_id&quot;:null,&quot;id&quot;:144192420,&quot;type&quot;:&quot;newsletter&quot;,&quot;reaction_count&quot;:0,&quot;comment_count&quot;:0,&quot;publication_id&quot;:null,&quot;publication_name&quot;:&quot;Calvin&quot;,&quot;publication_logo_url&quot;:&quot;https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb5ac185f-a363-4144-9bb9-fc5b1511b0e7_144x144.png&quot;,&quot;belowTheFold&quot;:true,&quot;youtube_url&quot;:null,&quot;show_links&quot;:null,&quot;feed_url&quot;:null}"></div><p>Here are the first four values</p><p><em><strong>hydra(1) = 1<br>hydra(2) = 3<br>hydra(3) = 11<br>hydra(4) = 1,114,111</strong></em></p><p>The exact answer to <em><strong>hydra(5)</strong></em> is:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;Let\\ F(x) = 2^x \\cdot (x + 2) - 1&quot;,&quot;id&quot;:&quot;CSEFDVVSHM&quot;}" data-component-name="LatexBlockToDOM"></div><p><em>To denote the n-th nesting of the function F:</em></p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;F^n(x) = \\underbrace{F(F(\\cdots F}_{n\\, \\text{times}}(x)\\cdots))\n&quot;,&quot;id&quot;:&quot;AIJIXYFCPV&quot;}" data-component-name="LatexBlockToDOM"></div><p>For convenience, let&#8217;s define a constant <em><strong>C</strong></em>:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;C = F^2(2 + 1) + 1 = F^2(3) + 1 = 22539988369408&quot;,&quot;id&quot;:&quot;ZPFUJUZJXJ&quot;}" data-component-name="LatexBlockToDOM"></div><p>The <em><strong>exact value</strong></em> of <em><strong>hydra(5)</strong></em> is given by:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;hydra(5) = 2 \\cdot F^{F^2(3) + 1}(F^2(3) + 1) + 1= 2 \\cdot F^C(C) + 1&quot;,&quot;id&quot;:&quot;DIEGVMMUSA&quot;}" data-component-name="LatexBlockToDOM"></div><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Calvin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h1>The Proof</h1><p>(Skip to Stage 4 for the math proof part!)</p><p>To discuss the evolution of the hydra throughout the hydra game, let&#8217;s define a state <code>([a, b, c, d, e], S)</code>, where the array represents the physical state of the hydra, and S is the step count. If <code>e &gt; 0</code>, then after 1 step, the state evolves to become <code>([a, b, c, d + S, e - 1], S + 1)</code>.</p><h2>Stage 1: Brute Force Coding</h2><p>For hydra game 1 - 4, I ran the following python code:</p><pre><code>def selectLevel(hydra: list) -&gt; int:
    for i in range(len(hydra)):
        # end of list, return the last level
        if (i + 1 == len(hydra)):
            if hydra[i] &gt; 0:
                return i
        # reached a node greater than 1, return
        if hydra[i] &gt; 1:
            return i
        # reached a level which is a leaf
        if hydra[i] == 1:
            if hydra[i + 1] == 0:
                return i
    raise Exception('bad selectLevel')

def simulate(hydra: list) -&gt; int:
    step = 0
    print(hydra)
    while hydra[0] &gt; 0:
        step += 1
        print(hydra, step)
        # find where to chop
        level = selectLevel(hydra)
        hydra[level] -= 1
        if level &gt; 0:
            hydra[level - 1] += step

def main():
    hydra_size = 4
    hydra = [1 for _ in range(hydra_size)]
    simulate(hydra)

main()</code></pre><p>Instead of explicitly modeling the tree, it&#8217;s equivalent to modeling a list where we ensure that we cannot decrement an array value to 0 unless its children (indicies to the right) are also 0.</p><p>For this, we comfortably simulated to demonstrate the sequence of 1, 3, 11, 1114111. For n=5, it started to take too long to run and it doesn&#8217;t seem like it&#8217;ll end anytime soon.</p><h2>Stage 2: Fast Chop</h2><p>The most expensive step is in pruning the large number of leaves of the root node. The numbers got ridiculously large and it spent a lot of time decrementing the root node, so I implemented Fast Chop:</p><p>Instead of decrementing one by one, I just decremented all but one of the root (since we can&#8217;t chop until all the other indicies have been cut to 0) and added the difference to the step count. Here&#8217;s the updated <code>simulate</code> function:</p><pre><code>def simulate(hydra: list, fast_chop: bool = False) -&gt; int:
    step = 0
    print(hydra)
    while hydra[0] &gt; 0:
        step += 1
        print(hydra, step)
        level = selectLevel(hydra)
        # chop
        if level == 0 and hydra[level] &gt; 1 and fast_chop:
            node_size = hydra[level]
            hydra[level] -= node_size - 1
            step += node_size - 2  # already incremented step, so - 2
        else:
            hydra[level] -= 1
            if level &gt; 0:
                hydra[level-1] += step
    return step - 1</code></pre><p>Still produced the right answers for 1 - 4.</p><p>For hydra(5), I quickly ran into this issue:</p><pre><code>ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit</code></pre><p>And not a lot of progress was made. It doesn&#8217;t seem like I can easily increase the limit to solve this problem&#8230; for reference this was the last thing I saw before this error:</p><pre><code>[5236751274087763604827335113503164542980300196051027772799054866553408315208749862621910699058066359304382851927240123690001657028076624243874905856369638573015266767173518533760881090665168657501466502575438814695379244920821816879398416379568995975331885966655627889373456252504294450808872324956279954441791424138238566544945055910878598449444356820527690288461439219089884273119672110780915086500181367389487847591622178596698590638598964162079073208267381018870683102042412374855218599064900594222409161345188381482756218697949512315491167618692787301782162184736991124223971701054644009826862190494573709271105283016638965612688564267675573446663137503920856946163800030385661836794421151038323102174273831660161943282774795456286219012025514744725129058365776715796186849330378082323946782828045123848071459942902828368123838185297691350796927183438726836331775328725893998022308339046666450649110098923213655788584420338021683310567449135992452580240598240949210098906606443034323449167094382342009720751240219585329918940963407988745009765182769239210185976272595207525201689610903562379503969853336779270227985605752034151825586837471803169417902911531690946991797181645549811951087317790371686134226956300584751700773029835508380819594560654781826332316405519852105346260214716258678990870714024313498644277768937813561794857945887410977910789497996279246650157863972578078236381769628727247654317441033159274592748068797421129624639669953783640862170282664397133282652299904463029437171264502226794590544077875216641287293069454710386248370680308708731029273018895879272852822220795151026965660328066845816623590817493823966958437300195250223341446166956531120990984133649788709888107516868282329681941441585518552792104733272698240012054637287259405459124930443017708422939350172006240588687592998904050506102490343772465687991422441448882011725609651783944361752435362062781589195510865475207181455225108775250001093738956361680441649527335822572336120455001762442505113797312189116632250263437321506263410518359810945829064312927586684675494831276823669068201460495423176123108409901698778301964724718957681497200352702701076000294559530265746359546619433943942940871442069738465827748357498595512514379055022004878344501765037265621885163003749999113603007286707281304209094602827386096346850598719467184159366295111010437092778471932435847585622290087202463029581238304221091731411342646010878158782598277736273725880547301418065127418837768153356358086680399888437754185935791653522527588641743823343406228041369298656125612209809308156697742068095146882731538744628536417045031881562234222022692975563640511286215378042016535361816522961162022289632861025679328602115231852923527050327616803340359251971379527588194183977451694394758300505712301729804885473523514141584822246161706088650448135123489998734583034450677447388385822893681353155622711404295181378827305822128040202984330793416842344269948350278999196273924686089626702081385958689896939223943945135658293948365266613362606866388013839023918573096791461760640816126897351506173851590079283933367950519316399171284435994198773170236082931563829489173596388230979952429868071244767722905954955193957444015662323635942083440563413775925250402785571597958667107947168757358479075225026526563032099728035575668109715823667685449996736046781140951726763715400819999867141471120936774055169955092818136292880667820574649938464671084088300302239619483984392792645074476910620078394019525809139132887110714887841277090342270471936907855105576556697910955352727191007370383437491260272759688502081997665341246371514518105005094429840370279355270165350942684119889898897785725295983426197612720256499277786884561502840067942542274821075820210267162117466922348246664073310562018052544726571554607306963521728197407669815297252788249623858769527282474860722408331998136164324865180469060271551771130264764731673686149466242204929133585865007342493911099519307953277164206639037770488051522091117052597342947125013900324244562835664605462604987945798805427063224160805807118493892312279199291896920341710392490804290716924090542886711173948988426531863920448490530479550509676455670400571503643438719913975901170479794011688362240795768686834708272261861187939120954315182187550884695934209489824329799121196304728207142986794926080, 22539988355169, 22539988369407, 0, 0] 5236751274087763604827335113503164542980300196051027772799054866553408315208749862621910699058066359304382851927240123690001657028076624243874905856369638573015266767173518533760881090665168657501466502575438814695379244920821816879398416379568995975331885966655627889373456252504294450808872324956279954441791424138238566544945055910878598449444356820527690288461439219089884273119672110780915086500181367389487847591622178596698590638598964162079073208267381018870683102042412374855218599064900594222409161345188381482756218697949512315491167618692787301782162184736991124223971701054644009826862190494573709271105283016638965612688564267675573446663137503920856946163800030385661836794421151038323102174273831660161943282774795456286219012025514744725129058365776715796186849330378082323946782828045123848071459942902828368123838185297691350796927183438726836331775328725893998022308339046666450649110098923213655788584420338021683310567449135992452580240598240949210098906606443034323449167094382342009720751240219585329918940963407988745009765182769239210185976272595207525201689610903562379503969853336779270227985605752034151825586837471803169417902911531690946991797181645549811951087317790371686134226956300584751700773029835508380819594560654781826332316405519852105346260214716258678990870714024313498644277768937813561794857945887410977910789497996279246650157863972578078236381769628727247654317441033159274592748068797421129624639669953783640862170282664397133282652299904463029437171264502226794590544077875216641287293069454710386248370680308708731029273018895879272852822220795151026965660328066845816623590817493823966958437300195250223341446166956531120990984133649788709888107516868282329681941441585518552792104733272698240012054637287259405459124930443017708422939350172006240588687592998904050506102490343772465687991422441448882011725609651783944361752435362062781589195510865475207181455225108775250001093738956361680441649527335822572336120455001762442505113797312189116632250263437321506263410518359810945829064312927586684675494831276823669068201460495423176123108409901698778301964724718957681497200352702701076000294559530265746359546619433943942940871442069738465827748357498595512514379055022004878344501765037265621885163003749999113603007286707281304209094602827386096346850598719467184159366295111010437092778471932435847585622290087202463029581238304221091731411342646010878158782598277736273725880547301418065127418837768153356358086680399888437754185935791653522527588641743823343406228041369298656125612209809308156697742068095146882731538744628536417045031881562234222022692975563640511286215378042016535361816522961162022289632861025679328602115231852923527050327616803340359251971379527588194183977451694394758300505712301729804885473523514141584822246161706088650448135123489998734583034450677447388385822893681353155622711404295181378827305822128040202984330793416842344269948350278999196273924686089626702081385958689896939223943945135658293948365266613362606866388013839023918573096791461760640816126897351506173851590079283933367950519316399171284435994198773170236082931563829489173596388230979952429868071244767722905954955193957444015662323635942083440563413775925250402785571597958667107947168757358479075225026526563032099728035575668109715823667685449996736046781140951726763715400819999867141471120936774055169955092818136292880667820574649938464671084088300302239619483984392792645074476910620078394019525809139132887110714887841277090342270471936907855105576556697910955352727191007370383437491260272759688502081997665341246371514518105005094429840370279355270165350942684119889898897785725295983426197612720256499277786884561502840067942542274821075820210267162117466922348246664073310562018052544726571554607306963521728197407669815297252788249623858769527282474860722408331998136164324865180469060271551771130264764731673686149466242204929133585865007342493911099519307953277164206639037770488051522091117052597342947125013900324244562835664605462604987945798805427063224160805807118493892312279199291896920341710392490804290716924090542886711173948988426531863920448490530479550509676455670400571503643438719913975901170479794011688362240795768686834708272261861187939120954315182187550884695934209489824329799121196304728207142986794926080</code></pre><p>Where the array is keeping track of the hydra state, and the last number is the step we were on.</p><h2>Stage 3: Figure out what we need to solve</h2><p>So I modified the <code>simulate</code> function again to figure out what is a tractable point to create a formula for the answer. I used stop_at to stop at a hydra of 3 levels deep (because that is empirically the point that the simulation was able to get to before it ran out of digits).</p><pre><code>def simulate(hydra: list, stop_at: int = 0, fast_chop: bool = False) -&gt; int:
    step = 0
    print(hydra)
    while hydra[0] &gt; 0:
        step += 1
        print(hydra, step) # , getSteps(hydra, step))
        if hydra[stop_at] == 0:
            return step
        level = selectLevel(hydra)
        # chop
        if level == 0 and hydra[level] &gt; 1 and fast_chop:
            node_size = hydra[level]
            hydra[level] -= node_size - 1
            step += node_size - 2
        else:
            hydra[level] -= 1
            if level &gt; 0:
                hydra[level-1] += step
    return step - 1</code></pre><p>For <code>stop_at = 3</code>, we get to the following state:</p><pre><code>([1, 1, 22539988369408, 0, 0], 22539988369408)</code></pre><p>So this hydra has 3 levels, with the third level having 22539988369408 heads at step 22539988369408. The fact that these two numbers are not really special, because when you make the first cut off of a linear tree [1, 1, 1, 1, &#8230;, 1, 1, 1] for step size S, the next step size is S + 1, and you add S siblings to the parent, so that level becomes S + 1. (by the way, when the simulation ran out of digits, the third level never made it past 1 minus that number &#128561;)</p><p>The goal is to create a closed form formula for the form: <code>([1, 1, c], S)</code>. Note it&#8217;s best not to constrain the problem where <code>c == S</code> because it&#8217;s easier to come up with the more general form and then plug in the situation where <code>c == S</code>.</p><h2>Stage 4: Inductive proof</h2><p>(note, I don&#8217;t know how to make writing LateX proofs in SubS NOT painful&#8230;)</p><p>Proof Strategy: Write formulas for simpler states, and then show how more complex states evolve into simpler states with solved formulas.</p><h3>Base Case ([1], S) &#8594; ([1 + A, 1], S)</h3><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(a)\\ ([1], S) \\rightarrow S;\\ \\text{on the S-th step, the last head is chopped}&quot;,&quot;id&quot;:&quot;KXXPNYUDXJ&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(b)\\ ([1 + A], S) \\rightarrow ([1], S + A) \\rightarrow S + A,\\ \\text{per (a)}&quot;,&quot;id&quot;:&quot;QTQQYJSZNN&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(c)\\ ([1, 1], S) \\to ([1 + S], S + 1]) \\to 2 \\cdot (S + 1) - 1,\\  \\text{per (b)}&quot;,&quot;id&quot;:&quot;KRLYEJGFZG&quot;}" data-component-name="LatexBlockToDOM"></div><p>writing <code>2*S + 1</code>  as  <code>2*(S + 1) - 1</code> for convenience as it will soon become clear.</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(d)\\ ([1 + A, 1], S) \\to ([1, 1], S + A]) \\to 2 \\cdot (S + A + 1) - 1,\\  \\text{per (c)}&quot;,&quot;id&quot;:&quot;EXGHRKEPGM&quot;}" data-component-name="LatexBlockToDOM"></div><h3>Solving for ([1 + A, 1 + B], S) and ([1, 1, 1], S)</h3><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(e)\\ ([1 + A, 1 + 1], S) \\to ([1 + S + A], 1], S + 1) \\to ([1, 1], 2 \\cdot S + A + 1) \\to 4 \\cdot (S + A + 1) - 1, \\text{per (d)}&quot;,&quot;id&quot;:&quot;TSKIROLSNB&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(f)\\ ([1 + A, 1 + 2], S) \\to ([1 + S + A], 1 + 1], S + 1) \\to ([1, 1 + 1], 2 \\cdot S + A + 1) \\to 8 \\cdot (S + A + 1) - 1, \\text{per (e)}&quot;,&quot;id&quot;:&quot;JJMXYMUVMY&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(g)\\ ([1 + A, 1 + 3], S) \\to ([1 + S + A], 1 + 2], S + 1) \\to ([1, 1 + 2], 2 \\cdot S + A + 1) \\to 16 \\cdot (S + A + 1) - 1, \\text{per (f)}&quot;,&quot;id&quot;:&quot;JFEUIRANYH&quot;}" data-component-name="LatexBlockToDOM"></div><p>Since the pattern is the same, therefore:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(h)\\ ([1 + A, 1 + B], S) \\to 2^{1 + B} \\cdot (S + A + 1) - 1, \\text{per (e), (f), &amp; (g)}&quot;,&quot;id&quot;:&quot;VRJDYKGGYW&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(i)\\ ([1, 1, 1], S) \\to ([1, 1 + S], S + 1) \\to 2^{S+1} \\cdot (S + 2) - 1 \\to 2 \\cdot (2^S \\cdot (S + 2) - 1) + 1, \\text{per (h)}&quot;,&quot;id&quot;:&quot;IQUQDNDLUC&quot;}" data-component-name="LatexBlockToDOM"></div><p>Recall that:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;Let\\ F(x) = 2^x \\cdot (x + 2) - 1&quot;,&quot;id&quot;:&quot;QTJNKKDYLU&quot;}" data-component-name="LatexBlockToDOM"></div><p>So:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(i)\\ ([1, 1, 1], S) \\to 2 \\cdot F(S) + 1&quot;,&quot;id&quot;:&quot;NSCHBUURDZ&quot;}" data-component-name="LatexBlockToDOM"></div><h2>Final Stretch: transform ([1, 1, X], S) &#8594; ([1, 1, 1], ???)</h2><p>Final stage of the proof is to find how to transform <em><strong>([1, 1, X], S)</strong></em> into <em><strong>([1, 1, 1], ???)</strong></em>, then input <em><strong>([1, 1, 22539988369408, 0, 0], 22539988369408)</strong></em> and <em>(i)</em><strong> </strong>to get the exact solution to<strong> </strong><em><strong>hydra(5)</strong></em>. First, we will solve <em><strong>([1, 1 + B, X], S)</strong></em> into <em><strong>([1, 1, X], S)</strong></em>.</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;\\text{Goal:}\\ ([1, 1+B, X], S) \\to ([1, 1, X], ????)&quot;,&quot;id&quot;:&quot;RWYAJGXPKI&quot;}" data-component-name="LatexBlockToDOM"></div><p>Inductively:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(j)\\ ([1, 1 + 1, X], S) \\to ([1 + S, 1, X], S + 1) \\to ([1, 1, X], 2 \\cdot (S + 1) - 1)&quot;,&quot;id&quot;:&quot;QHFZXUGEZP&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(k)\\ ([1, 1 + 2, X], S) \\to ([1 + S, 1 + 1, X], S + 1) \\to ([1, 1 + 1, X], 2 \\cdot (S + 1) - 1) \\to ([1, 1, 1], 2^2 \\cdot (S + 1) - 1, \\text{per (j)}&quot;,&quot;id&quot;:&quot;YUROBTUUSX&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(l)\\ ([1, 1 + B, X], S) \\to ([1, 1, X], 2^B \\cdot (S + 1) - 1), \\text{from (j) &amp; (k)}&quot;,&quot;id&quot;:&quot;YMAYQDUSPP&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(m)\\ ([1, 1, X], S) \\to ([1, 1 + S, X - 1], S + 1) \\to ([1, 1, X-1], 2^S \\cdot (S + 2) - 1), \\text{per (l)}&quot;,&quot;id&quot;:&quot;ARJVZRYSTN&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(m)\\ ([1, 1, X], S) \\to ([1, 1, X-1], 2^S \\cdot (S + 2) - 1) \\to ([1, 1, X-1], F(S))&quot;,&quot;id&quot;:&quot;QGLJMOHAOL&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(n)([1, 1, X], S) \\to ([1, 1, 1], F^{x-1}(S)), \\text{per (m)}&quot;,&quot;id&quot;:&quot;NOPQKQTNGW&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(o)([1, 1, X], S) \\to ([1, 1, 1], F^{x-1}(S)) \\to 2 \\cdot F^x(S), \\text{per (i) &amp; (n)}&quot;,&quot;id&quot;:&quot;SGMPCMNKHY&quot;}" data-component-name="LatexBlockToDOM"></div><p>For the case ([1, 1, S, 0, &#8230;], S), which will always be the case for the first instance of the height of any level as we discussed before (including 3):</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;(p)([1, 1, S], S) \\to  2 \\cdot F^S(S) + 1, \\text{per (o)}&quot;,&quot;id&quot;:&quot;FBISLDMPPN&quot;}" data-component-name="LatexBlockToDOM"></div><h3>Plugging in <strong>hydra(5)</strong> to the formula</h3><p>The answer is:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;([1, 1, 1, 1, 1], 1] \\to ([1, 1, C, 0, 0], C) \\to 2 \\cdot F^{C}(C))+ 1), \\text{where C=22539988369408}&quot;,&quot;id&quot;:&quot;YOXDHGLMUS&quot;}" data-component-name="LatexBlockToDOM"></div><h3>Validating with hydra(4)</h3><p>We know the answer to <em><strong>hydra(4)</strong></em> so we can validate our new formula with it. First step after <em><strong>([1, 1, 1, 1], 1)</strong></em> is <em><strong>([1, 1, 2, 0], 2)</strong></em>.</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;([1, 1, 2, 0], 2) \\to ([1, 1, 1, 0], 2^2 \\cdot (2 + 2) - 1) \\to ([1, 1, 1, 0], 15)&quot;,&quot;id&quot;:&quot;NVXBUVETSF&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;([1, 1, 1, 0], 15) \\to 2 \\cdot (2^{15} \\cdot (15 + 2) - 1) + 1 \\to 1114111&quot;,&quot;id&quot;:&quot;YWAIOQMLPB&quot;}" data-component-name="LatexBlockToDOM"></div><p>So the math checks out!</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Calvin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h1>Hinting of the larger pattern</h1><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!aYEC!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc2bd9d52-9172-41cb-b9f3-0c46aed27367_1024x1024.webp" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!aYEC!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc2bd9d52-9172-41cb-b9f3-0c46aed27367_1024x1024.webp 424w, https://substackcdn.com/image/fetch/$s_!aYEC!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc2bd9d52-9172-41cb-b9f3-0c46aed27367_1024x1024.webp 848w, https://substackcdn.com/image/fetch/$s_!aYEC!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc2bd9d52-9172-41cb-b9f3-0c46aed27367_1024x1024.webp 1272w, https://substackcdn.com/image/fetch/$s_!aYEC!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc2bd9d52-9172-41cb-b9f3-0c46aed27367_1024x1024.webp 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!aYEC!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc2bd9d52-9172-41cb-b9f3-0c46aed27367_1024x1024.webp" width="478" height="478" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/c2bd9d52-9172-41cb-b9f3-0c46aed27367_1024x1024.webp&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1024,&quot;width&quot;:1024,&quot;resizeWidth&quot;:478,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;A whimsical depiction of a Hydra in a scholarly setting. The Hydra, with multiple serpent-like heads, each wearing round spectacles, is surrounded by books and ancient scrolls. Each head is focused on a different book, suggesting a thirst for knowledge. The environment is a classic library with wooden shelves filled with books, dimly lit by overhead lamps casting a warm glow. The scene conveys a blend of mythical and intellectual charm, creating a unique, studious atmosphere.&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="A whimsical depiction of a Hydra in a scholarly setting. The Hydra, with multiple serpent-like heads, each wearing round spectacles, is surrounded by books and ancient scrolls. Each head is focused on a different book, suggesting a thirst for knowledge. The environment is a classic library with wooden shelves filled with books, dimly lit by overhead lamps casting a warm glow. The scene conveys a blend of mythical and intellectual charm, creating a unique, studious atmosphere." title="A whimsical depiction of a Hydra in a scholarly setting. The Hydra, with multiple serpent-like heads, each wearing round spectacles, is surrounded by books and ancient scrolls. Each head is focused on a different book, suggesting a thirst for knowledge. The environment is a classic library with wooden shelves filled with books, dimly lit by overhead lamps casting a warm glow. The scene conveys a blend of mythical and intellectual charm, creating a unique, studious atmosphere." srcset="https://substackcdn.com/image/fetch/$s_!aYEC!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc2bd9d52-9172-41cb-b9f3-0c46aed27367_1024x1024.webp 424w, https://substackcdn.com/image/fetch/$s_!aYEC!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc2bd9d52-9172-41cb-b9f3-0c46aed27367_1024x1024.webp 848w, https://substackcdn.com/image/fetch/$s_!aYEC!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc2bd9d52-9172-41cb-b9f3-0c46aed27367_1024x1024.webp 1272w, https://substackcdn.com/image/fetch/$s_!aYEC!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc2bd9d52-9172-41cb-b9f3-0c46aed27367_1024x1024.webp 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">This is hand-drawn by ChatGPT 4</figcaption></figure></div><p>I&#8217;m stopping here for now. I explored on a whim whether our magical function <em><strong>F(x)</strong></em> is related to the magical number <em><strong>22539988369408</strong></em> from <em><strong>hydra(5)</strong></em>. In fact, through brute force:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;F(39) + 1 = 2 ^ {39} * 41 - 1 + 1= 22539988369408&quot;,&quot;id&quot;:&quot;INIOFBXNDZ&quot;}" data-component-name="LatexBlockToDOM"></div><p>And even 39 is a function of <em><strong>F(x)</strong></em>:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;F(3) = 2^3 * 5  - 1 = 39&quot;,&quot;id&quot;:&quot;JINVCUROZU&quot;}" data-component-name="LatexBlockToDOM"></div><p>So:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;22539988369408 = F^2(3) + 1&quot;,&quot;id&quot;:&quot;ECFNXUPIOU&quot;}" data-component-name="LatexBlockToDOM"></div><p>In fact, you can rewrite all the previous answers as functions of F:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;hydra(1) = F(0) = 1&quot;,&quot;id&quot;:&quot;RXKGESUCNA&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;hydra(2) = 2 \\cdot F(0) + 1 = 3&quot;,&quot;id&quot;:&quot;AAFPIDFJXA&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;hydra(3) = 2 \\cdot F^1(1) + 1 = 11&quot;,&quot;id&quot;:&quot;SFTZMEIWUE&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;hydra(4) = 2 \\cdot F^2(2) + 1 = 1114111&quot;,&quot;id&quot;:&quot;DNAKJMVJYD&quot;}" data-component-name="LatexBlockToDOM"></div><p>Probably worth further investigating <em><strong>F(x)</strong></em> to understand how <em><strong>hydra(n)</strong></em> grows!</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/hydra5?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Thank you for reading Calvin. This post is public so feel free to share it.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/hydra5?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://blog.cal.vin/p/hydra5?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div>]]></content:encoded></item><item><title><![CDATA[Why do LLMs work?]]></title><description><![CDATA[Demystifying LLMs a non-technical explanation and simple examples]]></description><link>https://blog.cal.vin/p/why-do-llms-work</link><guid isPermaLink="false">https://blog.cal.vin/p/why-do-llms-work</guid><dc:creator><![CDATA[Calvin Li]]></dc:creator><pubDate>Sat, 13 Apr 2024 21:17:59 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Understanding <em><strong>why</strong></em> Large Language Models (LLMs) work is essential, but often obscured by technical explanation. This simple, non-technical guide aims to explain the foundational principles of their effectiveness. This understanding does not require delving into technical details or complex mathematics upon which LLMs are built. While the technical implementations are constantly evolving, the essence of this transformative technology remains constant and core to our foundational understanding.</p><p>LLMs are a groundbreaking technology, the biggest since arguably the internet and smartphones. It is important for the broader public to grasp the underpinnings of their effectiveness, as the technology is transformational. Yet, the very lack of a deeper understanding leads to widespread misconceptions and prevalent apprehensions, from claims of ineffectiveness to claims they&#8217;re Artificial General Intelligence (AGI) that will take over humanity. After this explanation, the hope is that you find neither of those claims reasonable.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Cal.vin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h1><em>What</em> are they?</h1><p>LLMs are &#8220;just&#8221; next word / token (tokens can be thought of as partial words) prediction machines.</p><p>More generally, LLMs are &#8220;just&#8221; pattern matching machines.</p><p><strong>LLMs demonstrate the extremes of what pattern matching machines are capable of.</strong></p><p>One realization is that many human jobs are 'just' pattern matching exercises. This is why it&#8217;s so important that most people should understand LLMs.</p><h1><em>How </em>do they work?</h1><p>The technical explanation of how LLMs work have been done well by many others, so I will not cover it. If you&#8217;re interested, I highly recommend this series by Youtuber Three Blue One Brown:</p><div id="youtube2-wjZofJX0v4M" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;wjZofJX0v4M&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/wjZofJX0v4M?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><h1>A Simple Example: Memorization</h1><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!0yfl!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!0yfl!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png 424w, https://substackcdn.com/image/fetch/$s_!0yfl!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png 848w, https://substackcdn.com/image/fetch/$s_!0yfl!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png 1272w, https://substackcdn.com/image/fetch/$s_!0yfl!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!0yfl!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png" width="624" height="415.4611398963731" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:514,&quot;width&quot;:772,&quot;resizeWidth&quot;:624,&quot;bytes&quot;:647703,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!0yfl!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png 424w, https://substackcdn.com/image/fetch/$s_!0yfl!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png 848w, https://substackcdn.com/image/fetch/$s_!0yfl!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png 1272w, https://substackcdn.com/image/fetch/$s_!0yfl!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e01eefa-eea3-431e-932f-8d0942002d87_772x514.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Here&#8217;s our first example to build our understanding of what LLMs are doing.</p><blockquote><p>Finish this sentence by filling in the blanks:<br><em><strong>&#8220;To be or not &lt;blank1&gt; &lt;blank2&gt;.&#8221;</strong></em></p></blockquote><p>The answer is: &#8220;to be&#8221;, completing the first line of Shakespeare&#8217;s most famous soliloquy from Hamlet.</p><p>By being able to answer this question correctly, there are already a few things we know about answerer, whether it&#8217;s human or machine:</p><ol><li><p>They must have had exposure to Shakespeare.</p></li><li><p>They have the ability to look at all the preceding words, and not just the last word, <em>&#8220;not&#8221;</em>.</p></li><li><p>They find the phrase to be notable and/or important, so that they are able to complete this sentence via free recall.</p></li><li><p>The start of the sentence (<em>&#8220;To be or not&#8221;)</em> must be sufficiently unique start to them.</p></li></ol><p><em>&#8220;To be or not to be&#8221;</em> is an example of the simplest pattern, a pattern with a single correct answer. These simple patterns are simply <em><strong>memorization</strong></em>. Another example of memorization-type patterns revolves around facts, e.g." &#8220;The capital of Japan is Tokyo&#8221;. Let's look at more complex patterns.</p><h1>A Simple Pattern</h1><p>Let&#8217;s look at things where there&#8217;s not a single right answer, but a number of plausible answers.</p><blockquote><p>Finish this sentence by filling in the blank:<br><em><strong>&#8220;Calvin&#8217;s favorite tropical fruit to eat is &lt;blank&gt;.&#8221;</strong></em></p></blockquote><p>Unlike the previous example, there are many possible answers to the sentence. An important place to start before exploring possible answers is to look at invalid answers:</p><ul><li><p><em>&#8220;rocks&#8221;</em> is not valid because it&#8217;s not edible.</p></li><li><p><em>&#8220;milk&#8221;</em> is not valid because it's not a fruit.</p></li><li><p><em>&#8220;blueberry&#8221;</em> is not valid because it&#8217;s not a tropical fruit.</p></li><li><p><em>&#8220;bananas&#8221;</em> are not correct because it&#8217;s not grammatically correct</p></li><li><p><em>&#8220;orenge&#8221;</em> or <em>&#8220;sldkfjsdklgha3u&#8221;</em> are not valid because they&#8217;re not words.</p></li><li><p><em>&#8220;very&#8221;</em> is not valid because it is not syntactically correct.</p></li></ul><h2>The pattern of grammar, syntax, and spelling</h2><p>The last three examples highlight one of the early accomplishments of LLMs, which is to be able to generate grammatically and syntactically correct sentences with proper spelling. Earlier models might have said a lot of nonsensical things, but they said grammatically and syntactically correct nonsensical things with proper spelling. A very important part of language levels are learning how to form correct sentences. While it's possible to enumerate all the rules of the English language, what is done by most people and also by language models, is to apply patterns (e.g. sometimes singular subject is followed by a verb the ends in s) while also memorizing exceptions instead of checking all enumerated rules for each word in their sentence.</p><h2>Implicit truths</h2><p>In the space of possible answers, there's a whole spectrum of likely and unlikely answers. There are some really important and interesting features in the unlikely yet possible answers:</p><ul><li><p><em>&#8220;ambrosia&#8221;</em> is unlikely because it&#8217;s a fictional fruit from mythology, and there&#8217;s no context suggesting that Calvin is in a fictional universe, that Calvin is looking for something fictional, or that Calvin is capable of enjoying eating fictional fruits.</p></li><li><p><em>&#8220;manchineel&#8221;</em> is unlikely because it&#8217;s highly toxic to humans, and there&#8217;s no context suggesting that Calvin is not human.</p></li><li><p><em>&#8220;orange&#8221;</em> is an unlikely answer because there&#8217;s widespread misconception that it&#8217;s a tropical fruit, when it&#8217;s actually subtropical. It&#8217;s technically wrong, but colloquially not.</p></li><li><p><em>&#8220;avocado&#8221;</em> is unlikely because most people don&#8217;t consider them to be a fruit, even though they&#8217;re technically a fruit.</p></li><li><p><em>&#8220;soursop&#8221;</em> is an unlikely answer because it&#8217;s rare and most people don&#8217;t know of the fruit (even though it&#8217;s actually my favorite fruit).</p></li></ul><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!VFMg!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b5ea6f0-3c18-4f29-90db-5d9389403350_602x345.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!VFMg!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b5ea6f0-3c18-4f29-90db-5d9389403350_602x345.png 424w, https://substackcdn.com/image/fetch/$s_!VFMg!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b5ea6f0-3c18-4f29-90db-5d9389403350_602x345.png 848w, https://substackcdn.com/image/fetch/$s_!VFMg!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b5ea6f0-3c18-4f29-90db-5d9389403350_602x345.png 1272w, https://substackcdn.com/image/fetch/$s_!VFMg!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b5ea6f0-3c18-4f29-90db-5d9389403350_602x345.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!VFMg!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b5ea6f0-3c18-4f29-90db-5d9389403350_602x345.png" width="372" height="213.18936877076413" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/1b5ea6f0-3c18-4f29-90db-5d9389403350_602x345.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:345,&quot;width&quot;:602,&quot;resizeWidth&quot;:372,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!VFMg!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b5ea6f0-3c18-4f29-90db-5d9389403350_602x345.png 424w, https://substackcdn.com/image/fetch/$s_!VFMg!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b5ea6f0-3c18-4f29-90db-5d9389403350_602x345.png 848w, https://substackcdn.com/image/fetch/$s_!VFMg!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b5ea6f0-3c18-4f29-90db-5d9389403350_602x345.png 1272w, https://substackcdn.com/image/fetch/$s_!VFMg!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b5ea6f0-3c18-4f29-90db-5d9389403350_602x345.png 1456w" sizes="100vw" loading="lazy"></picture><div></div></div></a><figcaption class="image-caption">A soursop</figcaption></figure></div><p>The first two examples implicit assumptions in the sentence completion task. The latter highlights, statistically unlikely answers, based on &#8220;common knowledge&#8221; of the population of English speakers. These examples are technically correct as they are all tropical fruits, but unlikely in different ways.</p><h2>What is the real answer?</h2><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Ygb6!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fafd2aed6-6b4a-4d36-8b0f-7bf1ecf635b8_236x234.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Ygb6!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fafd2aed6-6b4a-4d36-8b0f-7bf1ecf635b8_236x234.png 424w, https://substackcdn.com/image/fetch/$s_!Ygb6!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fafd2aed6-6b4a-4d36-8b0f-7bf1ecf635b8_236x234.png 848w, https://substackcdn.com/image/fetch/$s_!Ygb6!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fafd2aed6-6b4a-4d36-8b0f-7bf1ecf635b8_236x234.png 1272w, https://substackcdn.com/image/fetch/$s_!Ygb6!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fafd2aed6-6b4a-4d36-8b0f-7bf1ecf635b8_236x234.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Ygb6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fafd2aed6-6b4a-4d36-8b0f-7bf1ecf635b8_236x234.png" width="236" height="234" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/afd2aed6-6b4a-4d36-8b0f-7bf1ecf635b8_236x234.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:234,&quot;width&quot;:236,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Ygb6!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fafd2aed6-6b4a-4d36-8b0f-7bf1ecf635b8_236x234.png 424w, https://substackcdn.com/image/fetch/$s_!Ygb6!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fafd2aed6-6b4a-4d36-8b0f-7bf1ecf635b8_236x234.png 848w, https://substackcdn.com/image/fetch/$s_!Ygb6!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fafd2aed6-6b4a-4d36-8b0f-7bf1ecf635b8_236x234.png 1272w, https://substackcdn.com/image/fetch/$s_!Ygb6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fafd2aed6-6b4a-4d36-8b0f-7bf1ecf635b8_236x234.png 1456w" sizes="100vw" loading="lazy"></picture><div></div></div></a></figure></div><p>To determine the best answers to &#8220;<em><strong>Calvin&#8217;s favorite tropical fruit to eat is &lt;blank&gt;</strong></em>&#8221;, we could conduct a survey. We would ask people named Calvin about their favorite tropical fruit, and compile the answers. The most frequently chosen fruits would provide the possible real answers to our question.</p><p>For instance, if we surveyed 100 Calvins and found that 40 of them prefer bananas, 25 mangoes, 15 pineapples, 12 papayas, and 8 coconuts, these fruits would form our list of top answers, ranked by popularity. This method doesn't tell us why these fruits are preferred or their exact percentages among all Calvins worldwide, but it gives us a good snapshot:</p><ol><li><p><em>&#8220;banana&#8221; - 40%</em></p></li><li><p><em>&#8220;mango&#8221; - 25%</em></p></li><li><p><em>&#8220;pineapple&#8221; - 15%</em></p></li><li><p><em>&#8220;papaya&#8221; - 12%</em></p></li><li><p><em>&#8220;coconut&#8221; - 8%</em></p></li></ol><p>This approach of collecting and analyzing responses helps illustrate how LLMs process language. They evaluate vast amounts of data and predict the most likely next word or phrase based on patterns they have observed in the data they were trained on, similar to how we've determined the favorite fruits from our survey results.</p><p>To fill in the blank of our question, LLMs would sample from the distributions of possible answers, and pick one at random based on the weighting of each word (i.e. <em>&#8220;banana&#8221; - 40%</em> would more likely be chosen than <em>&#8220;papaya&#8221; - 12%</em>).</p><h1>Inverting the Question</h1><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!930x!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae54e6bd-f2f9-4725-964f-6295717c3622_594x632.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!930x!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae54e6bd-f2f9-4725-964f-6295717c3622_594x632.png 424w, https://substackcdn.com/image/fetch/$s_!930x!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae54e6bd-f2f9-4725-964f-6295717c3622_594x632.png 848w, https://substackcdn.com/image/fetch/$s_!930x!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae54e6bd-f2f9-4725-964f-6295717c3622_594x632.png 1272w, https://substackcdn.com/image/fetch/$s_!930x!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae54e6bd-f2f9-4725-964f-6295717c3622_594x632.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!930x!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae54e6bd-f2f9-4725-964f-6295717c3622_594x632.png" width="234" height="248.96969696969697" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/ae54e6bd-f2f9-4725-964f-6295717c3622_594x632.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:632,&quot;width&quot;:594,&quot;resizeWidth&quot;:234,&quot;bytes&quot;:178905,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!930x!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae54e6bd-f2f9-4725-964f-6295717c3622_594x632.png 424w, https://substackcdn.com/image/fetch/$s_!930x!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae54e6bd-f2f9-4725-964f-6295717c3622_594x632.png 848w, https://substackcdn.com/image/fetch/$s_!930x!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae54e6bd-f2f9-4725-964f-6295717c3622_594x632.png 1272w, https://substackcdn.com/image/fetch/$s_!930x!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae54e6bd-f2f9-4725-964f-6295717c3622_594x632.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>To begin appreciating the nuance and complexity of LLMs, it&#8217;s worth asking the inverse question to <em>&#8220;Calvin&#8217;s favorite tropical fruit to eat is &lt;blank&gt;&#8221;</em>, which is to find the all questions that share the same distribution of possible answers: 1) <em>&#8220;banana&#8221;</em> 2) <em>&#8220;mango&#8221;</em> 3) <em>&#8220;pineapple&#8221;</em> 4) <em>&#8220;papaya&#8221;, </em>5) <em>&#8220;coconut&#8221;</em>.</p><blockquote><p>What are questions that yield the <em><strong>same</strong></em> distribution of responses to:<br><em><strong>&#8220;Calvin&#8217;s favorite tropical fruit to eat is &lt;blank&gt;.&#8221;</strong></em></p></blockquote><p>Here are some questions that is highly likely to have the same distribution:</p><ul><li><p><em><strong>&#8220;Calvin&#8217;s favorite fruit from tropical areas to eat is &lt;blank&gt;.&#8221;</strong></em></p></li><li><p><em><strong>&#8220;The tropical fruit that is Calvin&#8217;s favorite to eat is &lt;blank&gt;.&#8221;</strong></em></p></li><li><p><em><strong>&#8220;Calvin&#8217;s number 1 tropical fruit that he eats is &lt;blank&gt;.&#8221;</strong></em></p></li><li><p><em><strong>&#8220;Calvin&#8217;s fruit from regions of the world near the equator that he enjoys eating the most is &lt;blank&gt;.&#8221;</strong></em></p></li></ul><p>Here are versions where the &lt;blank&gt; is not at the end, which is valid:</p><ul><li><p><em><strong>&#8220;&lt;blank&gt; is Calvin&#8217;s favorite tropical fruit to eat.&#8221;</strong></em></p></li><li><p><em><strong>&#8220;Calvin favorite tropical fruit to eat is &lt;blank&gt;.&#8221;</strong></em></p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/why-do-llms-work?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Thank you for reading Cal.vin! This post is public so feel free to share it.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/why-do-llms-work?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://blog.cal.vin/p/why-do-llms-work?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div></li></ul><h2>LLMs (and you) understanding of the essence of the question</h2><p>Any system that is capable of providing the same distributions to all the variations of the same question demonstrates the ability to capture the core essence of the sentence. LLMs are such a system, ignoring some of the randomness inherent in the technical implementation details.</p><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!ec7F!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ecf9e20-19cf-4738-a430-92930e50ce95_966x232.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!ec7F!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ecf9e20-19cf-4738-a430-92930e50ce95_966x232.png 424w, https://substackcdn.com/image/fetch/$s_!ec7F!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ecf9e20-19cf-4738-a430-92930e50ce95_966x232.png 848w, https://substackcdn.com/image/fetch/$s_!ec7F!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ecf9e20-19cf-4738-a430-92930e50ce95_966x232.png 1272w, https://substackcdn.com/image/fetch/$s_!ec7F!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ecf9e20-19cf-4738-a430-92930e50ce95_966x232.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!ec7F!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ecf9e20-19cf-4738-a430-92930e50ce95_966x232.png" width="526" height="126.3271221532091" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/0ecf9e20-19cf-4738-a430-92930e50ce95_966x232.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:232,&quot;width&quot;:966,&quot;resizeWidth&quot;:526,&quot;bytes&quot;:38603,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!ec7F!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ecf9e20-19cf-4738-a430-92930e50ce95_966x232.png 424w, https://substackcdn.com/image/fetch/$s_!ec7F!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ecf9e20-19cf-4738-a430-92930e50ce95_966x232.png 848w, https://substackcdn.com/image/fetch/$s_!ec7F!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ecf9e20-19cf-4738-a430-92930e50ce95_966x232.png 1272w, https://substackcdn.com/image/fetch/$s_!ec7F!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ecf9e20-19cf-4738-a430-92930e50ce95_966x232.png 1456w" sizes="100vw" loading="lazy"></picture><div></div></div></a></figure></div><p>By the definition of &#8220;to understand&#8221;, such a system must <em><strong>understand</strong></em> the sentence in order to reliably and predictably produce the same distribution of possible answers under all variations that have the same inherent meaning.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Calvin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h2>Analyzing <em>similar</em> distributions</h2><p>Let&#8217;s explore questions where the distributions would be similar, but probably not the same.</p><ul><li><p><em><strong>&#8220;Calvin&#8217;s favorite fruit is &lt;blank&gt;.&#8221;</strong></em></p><ul><li><p>There are non-tropical fruits that will enter the distribution, e.g. &#8220;<em>blueberries&#8221;</em>.</p></li></ul></li><li><p><em><strong>&#8220;Jonathan&#8217;s favorite tropical fruit to eat is &lt;blank&gt;.&#8221;</strong></em></p><ul><li><p>There might be some correlation with names and favorite tropical fruit to eat.</p></li></ul></li><li><p><em><strong>&#8220;Catherine&#8217;s favorite tropical fruit to eat is &lt;blank&gt;.&#8221;</strong></em></p><ul><li><p>There might be a gender correlation, which is more likely than name-only.</p></li></ul></li><li><p><em><strong>&#8220;Kiriyama&#8217;s favorite tropical tropical fruit to eat is &lt;blank&gt;.&#8221;</strong></em></p><ul><li><p>There&#8217;s definitely a regional correlation with those names, and there&#8217;s regional correlations with food preferences.</p></li></ul></li><li><p><em><strong>&#8220;Calvin&#8217;s favorite fruit to eat is &lt;blank&gt;, which is a tropical fruit.&#8221;</strong></em></p><p><em><strong>&#8220;Calvin&#8217;s favorite food to eat is &lt;blank&gt;, which is a tropical fruit.&#8221;</strong></em></p><ul><li><p>This is a bit nuanced, as the distributions of Calvin&#8217;s whose favorite food/fruit happens to be a tropical fruit might be different than only looking at the favorite tropical fruit of Calvin&#8217;s.</p></li></ul></li></ul><p>Analyzing these subtle differences demonstrates that successful LLMs must encapsulate these observed statistical nuanced of the real-world into their next word / token prediction capabilities.</p><h1>Longer responses</h1><h2>Longer responses: Memorization</h2><p>Let&#8217;s return to the Shakespeare example. At each step, we will ask the LLM to only generate the next word (let&#8217;s simplify and ignore tokens):</p><blockquote><p>Finish the sentence <em><strong>&#8220;To be or not&#8221;</strong></em>:<em><strong><br>&#8220;To be or not&#8221; &#8594; &#8220; to&#8221;</strong></em><br><em><strong>&#8220;To be or not to&#8221; &#8594; &#8220; be&#8221;<br>&#8220;To be or not to be&#8221; &#8594; &#8220;, that&#8221;<br>&#8220;To be or not to be, that&#8221; &#8594; &#8220; is&#8221;<br>&#8220;To be or not to be, that is&#8221; &#8594; &#8220; the&#8221;<br>&#8220;To be or not to be, that is the&#8221; &#8594; &#8220; question&#8221;<br>&#8220;To be or not to be, that is the question&#8221; &#8594; &#8220;.&#8221;<br>&#8220;To be or not to be, that is the question.&#8221;</strong></em></p></blockquote><p>Each step is generating the next word, but given the constraints of the input (in this <em><strong>memorization</strong></em>) task, there is only one right answer, so repeated applications will produce the rest of Shakespeare&#8217;s Hamlet if the LLM decides that it is important enough and unique enough (we will discuss this in the next section of Memorization &#8594; Generalization).</p><h2>Longer responses: Simple Pattern</h2><p>Let&#8217;s return to discussing my favorite tropical fruit. Similar to before, we will ask the LLM to only generate the next word, but since it&#8217;s not a notable literature like Shakespeare, the generations will have more randomness to it.</p><blockquote><p>Finish the sentence <em><strong>&#8220;Calvin&#8217;s favorite tropical fruit is&#8221;</strong></em>:<em><strong><br>&#8220;Calvin&#8217;s favorite tropical fruit is&#8221;<br>          &#8594; </strong>1. &#8220;banana&#8221; 2. &#8220;mango&#8221;<strong> 3. &#8220;pineapple&#8221;<br>&#8220;Calvin&#8217;s favorite tropical fruit is pineapple&#8221;<br>          &#8594; </strong>1. &#8220;!&#8221;<strong> 2. &#8220; , despite&#8221; </strong>3. &#8220; , despite&#8221;<strong><br>&#8220;&#8230; tropical fruit is pineapple, despite&#8221;<br>          &#8594; 1. &#8220; their&#8221;</strong> 2. &#8220; how&#8221; 3. &#8220; his&#8221;<strong><br>&#8220;&#8230; tropical fruit is pineapple, despite their&#8221; <br>          &#8594; 1. &#8220; spikiness&#8221;</strong> 2. &#8220; roughness&#8221; 3. &#8220; appearance&#8221;<br><strong>&#8220;&#8230; tropical fruit is pineapple, despite their spikiness&#8221; <br>          &#8594; </strong>1. &#8220;.&#8221; <strong>2. &#8220;!&#8221;</strong> 3. &#8220;, showcasing&#8221;<br><strong>&#8220;Calvin's favorite tropical fruit is pineapple, despite their spikiness!&#8221;</strong></em></p></blockquote><p>At each stage of generating the next word (token), the system produces a probability distribution of the top possible candidates, and then the system selects one at random, with the top choices being more likely to be selected than the non-top choices. The system can be configured to have no randomness, and only pick the top option, or pick all the possible candidates in complete randomness. It&#8217;s been observed that somewhere in the middle has the best outcome.</p><h1>Memorization&#8594;Generalization, Compression, and Long Context</h1><h2>Memorization &#8594; Generalization</h2><p>A system whose goals are to produce next token prediction will likely na&#239;vely <em><strong>memorize</strong></em> the solution.</p><p>Provided with enough data, the system is forced to <em><strong>generalize</strong></em>, and if executed well with high quality data, they may do it well.</p><p>An observation with building LLMs is that they go through the memorization to generalization transition at some critical amount of data. While some tasks still require memorization, e.g. notable quotes and facts, the generalization of information is how LLMs learn to understand through complex patterns.</p><h2>Generalization as compression</h2><p>The generalization is what is often referred to as <em><strong>compression</strong></em> of real world information. The explicit goal of a lot of prior research in this space was around how to minimally store the rich information found throughout the world.</p><h2>Why do we need such large models?</h2><p>A system&#8217;s ability to produce complex patterns depends largely on its size. There are two core measures of its size, the first is the size of the model, and the second is the amount of data fed into it.</p><p><em><strong>For system size:</strong></em></p><ul><li><p>A small system is incapable of encapsulate many patterns or complex patterns.</p></li><li><p>A large system is capable of encapsulating many complex patterns.</p></li><li><p>An arbitrarily large system is capable of encapsulating patterns of arbitrary complexity.</p></li></ul><p><em><strong>For data size:</strong></em></p><ul><li><p>A system with very little data is able to learn very little</p></li><li><p>A system with a lot of data, but poor quality data or data with not a lot of information (e.g. repetitive), also learns very little</p></li><li><p>A system with diverse, high quality, and information rich data learns a lot</p></li></ul><p>The empirical observations are that these systems exhibit <a href="https://arxiv.org/abs/2001.08361">power-law scaling with respect to increases in system size, data size, and compute resources</a>, which has fueled the race among tech giants and well-funded startups to spend billions of dollars building bigger and more powerful models. The training cost for just <a href="https://www.wired.com/story/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over/">OpenAI&#8217;s ChatGPT4 exceeds $100 million</a>.</p><h2>Conceptually Simple: Long Inputs</h2><p>Long inputs only require that the LLM be set up to accept the longer inputs, a.k.a. <em>Context Length</em>. In the current implementation, there is a limit to how long they can be, but the limits are getting really large (Google&#8217;s Gemini&#8217;s acceptable context lengths exceeds the length of all the <em>Harry Potter</em> books.</p><h1>Mixing Patterns</h1><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!fasf!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcec2585d-d4a5-420f-aa9f-c1d12aa4c313_1245x700.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!fasf!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcec2585d-d4a5-420f-aa9f-c1d12aa4c313_1245x700.png 424w, https://substackcdn.com/image/fetch/$s_!fasf!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcec2585d-d4a5-420f-aa9f-c1d12aa4c313_1245x700.png 848w, https://substackcdn.com/image/fetch/$s_!fasf!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcec2585d-d4a5-420f-aa9f-c1d12aa4c313_1245x700.png 1272w, https://substackcdn.com/image/fetch/$s_!fasf!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcec2585d-d4a5-420f-aa9f-c1d12aa4c313_1245x700.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!fasf!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcec2585d-d4a5-420f-aa9f-c1d12aa4c313_1245x700.png" width="560" height="314.859437751004" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/cec2585d-d4a5-420f-aa9f-c1d12aa4c313_1245x700.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:700,&quot;width&quot;:1245,&quot;resizeWidth&quot;:560,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!fasf!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcec2585d-d4a5-420f-aa9f-c1d12aa4c313_1245x700.png 424w, https://substackcdn.com/image/fetch/$s_!fasf!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcec2585d-d4a5-420f-aa9f-c1d12aa4c313_1245x700.png 848w, https://substackcdn.com/image/fetch/$s_!fasf!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcec2585d-d4a5-420f-aa9f-c1d12aa4c313_1245x700.png 1272w, https://substackcdn.com/image/fetch/$s_!fasf!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcec2585d-d4a5-420f-aa9f-c1d12aa4c313_1245x700.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">A picture of Einstein in the style of Van Gogh</figcaption></figure></div><p>What happens when you combine multiple different next-token and join the distributions to produce the next token / word task?</p><p>The result is that they produce surprisingly good results of the intersection between multiple patterns (each of the concepts bolded for emphasis).</p><blockquote><p><strong>Prompt: </strong><em>&#8220;Write a <strong>metered sonnet</strong> about <strong>Materials Science</strong> in the <strong>style of Shakespeare</strong>&#8221;</em></p><p>Upon the anvil of the world so vast,<br>In forge of flame and intellect combined,<br>The craft of Materials Science cast,<br>A realm where art and physics are entwined.</p><p>With metals, polymers, ceramics bright,<br>Each substance shaped by hands both firm and wise,<br>Their structures morph from darkness into light,<br>Beneath the keen Materials Scientist's eyes.</p><p>In lattice bound, the atoms dance and play,<br>Their secrets told in whispers soft and clear,<br>From carbon's bond to silicon's array,<br>The elements their mysteries unveil.</p><p>So let us hail this noble, silent quest,<br>Where through Materials Science, we're truly blessed.</p></blockquote><p>This is a simplification of what actually happens within LLMs, but the intuition is the same: LLMs are extremely adept at blending multitudes of patterns in novel ways.</p><h1>Bias in LLMs</h1><p>LLMs are simply learning patterns in the data provided to the system. There&#8217;s an old saying in machine learning, which is <em><strong>&#8220;Garbage In, Garbage Out&#8221;</strong></em>. If the training data is garbage, then the output will also be garbage. Similarly, there&#8217;s <em><strong>&#8220;racism in, racism out&#8221;</strong></em>, in that if you train the LLMs with racist content, then the produced sentences will be racist.</p><p>It&#8217;s extremely difficult to tease out the subtle forms of discrimination that statistically exists in the dataset. Without going into it too deeply, this is an area rife with issues of <a href="https://en.wikipedia.org/wiki/Statistical_discrimination_(economics)">Statistical Discrimination (wikipedia)</a>.</p><h1>Prompt Sensitivity</h1><p>Different models, e.g. ChatGPT, Gemini, Claude, etc. are sensitive to how you &#8220;talk&#8221; to it (a.k.a. prompting) because each of the models are trained and tuned with different data sources that have gone through various custom refinements to specific styles. This is done on the model&#8217;s part to ensure more consistent training and quality of the model, and therefore requires users to &#8220;talk&#8221; / interact to them in different manners.</p><h2>&#8220;show your work&#8221; &amp; &#8220;tipping&#8221;</h2><p>It&#8217;s been thoroughly demonstrated that asking LLMs to &#8220;show your work&#8221; or &#8220;offer tips&#8221; improves model performance. Is it because these models are sentient?</p><p><strong>No.</strong></p><p>It&#8217;s just a reflection of the training data. It&#8217;s trained off of human generated information, and higher quality responses are associated with questions asking for thoroughness (&#8220;show your work&#8221;) or provide extrinsic motivation (&#8220;tipping&#8221;). The models goals are to be able to predict and match the behavior form its training data, which is all human psychology.</p><h2>Prompt Engineering a.k.a. High EQ Engineers</h2><p>In order to be adept at working with these language models, the users need high EQ, because the systems are built to reflect human emotions, thinking, and reasoning. Fundamentally, the way to interact with these systems require empathy for how humans would respond, and therefore requires skillsets often unassociated with engineers (though great engineers have always required great EQ).</p><p>I understand that a profession that requires primarily writing paragraphs of English is a weird way to think about &#8220;engineering&#8221;, but I think the title is appropriate given that they are experts on working with these systems, similar to how a python programmer is proficient in writing in a way that makes the most sense to the python programming language.</p><h1>LLMs understand, but do not reason</h1><p><em>Caveat: This section addresses the limitation of LLMs, but these limitations are reflective of the <strong>current</strong> technology, not necessarily the limitations of all future language models; after all, humans are also language models (and many other things as well).</em></p><p>The patterns that LLMs are able to encapsulate and emulate are extremely complex, and an arbitrarily large and well trained system will be able to emulate arbitrarily large and complex and patterns. We have established this to be a form of understanding. However, there are really important limitations to the technology that we need to understand, namely that LLMs cannot perform computation, or analogously, LLMs do not reason.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Cal.vin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h2>Pattern Matching / Understanding: Chess &#9989;</h2><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!rITW!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49443826-8be0-4a26-b919-8f7d524e8dd0_720x720.gif" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!rITW!,w_424,c_limit,f_webp,q_auto:good,fl_lossy/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49443826-8be0-4a26-b919-8f7d524e8dd0_720x720.gif 424w, https://substackcdn.com/image/fetch/$s_!rITW!,w_848,c_limit,f_webp,q_auto:good,fl_lossy/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49443826-8be0-4a26-b919-8f7d524e8dd0_720x720.gif 848w, https://substackcdn.com/image/fetch/$s_!rITW!,w_1272,c_limit,f_webp,q_auto:good,fl_lossy/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49443826-8be0-4a26-b919-8f7d524e8dd0_720x720.gif 1272w, https://substackcdn.com/image/fetch/$s_!rITW!,w_1456,c_limit,f_webp,q_auto:good,fl_lossy/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49443826-8be0-4a26-b919-8f7d524e8dd0_720x720.gif 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!rITW!,w_1456,c_limit,f_auto,q_auto:good,fl_lossy/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49443826-8be0-4a26-b919-8f7d524e8dd0_720x720.gif" width="406" height="406" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/49443826-8be0-4a26-b919-8f7d524e8dd0_720x720.gif&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:720,&quot;width&quot;:720,&quot;resizeWidth&quot;:406,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;CDN media&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="CDN media" title="CDN media" srcset="https://substackcdn.com/image/fetch/$s_!rITW!,w_424,c_limit,f_auto,q_auto:good,fl_lossy/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49443826-8be0-4a26-b919-8f7d524e8dd0_720x720.gif 424w, https://substackcdn.com/image/fetch/$s_!rITW!,w_848,c_limit,f_auto,q_auto:good,fl_lossy/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49443826-8be0-4a26-b919-8f7d524e8dd0_720x720.gif 848w, https://substackcdn.com/image/fetch/$s_!rITW!,w_1272,c_limit,f_auto,q_auto:good,fl_lossy/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49443826-8be0-4a26-b919-8f7d524e8dd0_720x720.gif 1272w, https://substackcdn.com/image/fetch/$s_!rITW!,w_1456,c_limit,f_auto,q_auto:good,fl_lossy/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49443826-8be0-4a26-b919-8f7d524e8dd0_720x720.gif 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">OpenAI&#8217;s ChatGPT 4 (Black) vs Stockfish 15 (White) source: chess.com</figcaption></figure></div><p>LLMs are capable of learning the complex space of chess because it has been given enough data to learn about the structure of chess, what are valid moves, similar to how LLMs have learned about the complex structure of English grammar (and what are valid words at which time). It also helps that there is so much content related to chess that the language model can learn from.</p><h2>Computation / Reasoning: Multiplication &#128078;</h2><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!d8Av!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F250ac519-ff1a-4508-a0b7-303be59c211d_1138x406.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!d8Av!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F250ac519-ff1a-4508-a0b7-303be59c211d_1138x406.png 424w, https://substackcdn.com/image/fetch/$s_!d8Av!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F250ac519-ff1a-4508-a0b7-303be59c211d_1138x406.png 848w, https://substackcdn.com/image/fetch/$s_!d8Av!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F250ac519-ff1a-4508-a0b7-303be59c211d_1138x406.png 1272w, https://substackcdn.com/image/fetch/$s_!d8Av!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F250ac519-ff1a-4508-a0b7-303be59c211d_1138x406.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!d8Av!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F250ac519-ff1a-4508-a0b7-303be59c211d_1138x406.png" width="502" height="179.09666080843584" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/250ac519-ff1a-4508-a0b7-303be59c211d_1138x406.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:406,&quot;width&quot;:1138,&quot;resizeWidth&quot;:502,&quot;bytes&quot;:57406,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!d8Av!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F250ac519-ff1a-4508-a0b7-303be59c211d_1138x406.png 424w, https://substackcdn.com/image/fetch/$s_!d8Av!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F250ac519-ff1a-4508-a0b7-303be59c211d_1138x406.png 848w, https://substackcdn.com/image/fetch/$s_!d8Av!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F250ac519-ff1a-4508-a0b7-303be59c211d_1138x406.png 1272w, https://substackcdn.com/image/fetch/$s_!d8Av!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F250ac519-ff1a-4508-a0b7-303be59c211d_1138x406.png 1456w" sizes="100vw" loading="lazy"></picture><div></div></div></a><figcaption class="image-caption">The correct answer is <strong>293,857,235 x 235,235 = 69,125,506,675,225</strong></figcaption></figure></div><p>What is easier than chess to most, but something LLMs cannot do, is long multiplication. Short and simple multiplication (6 x 7 = 42) is something that LLMs can memorize, and there are simpler patterns that it can learn. Long multiplication is a domain where the data is so dense that it&#8217;s impossible for a finitely sized system to memorize all the permutations. The only efficient methods are to implement one of the many <a href="https://en.wikipedia.org/wiki/Multiplication_algorithm">multiplication algorithms</a>, which has complex recurring structures, which na&#239;vely LLMs cannot handle.</p><p>In the illustrations, the LLMs were able to predict that the result is a number, the right number of digits, the fact that the number starts with 69, and ends in a 5, but struggled to come up with the right answer.</p><h2>Tool Use</h2><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!uJDE!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5062a702-aee9-442d-9a5a-3bd87355432c_1138x406.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!uJDE!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5062a702-aee9-442d-9a5a-3bd87355432c_1138x406.png 424w, https://substackcdn.com/image/fetch/$s_!uJDE!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5062a702-aee9-442d-9a5a-3bd87355432c_1138x406.png 848w, https://substackcdn.com/image/fetch/$s_!uJDE!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5062a702-aee9-442d-9a5a-3bd87355432c_1138x406.png 1272w, https://substackcdn.com/image/fetch/$s_!uJDE!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5062a702-aee9-442d-9a5a-3bd87355432c_1138x406.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!uJDE!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5062a702-aee9-442d-9a5a-3bd87355432c_1138x406.png" width="484" height="172.6748681898067" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/5062a702-aee9-442d-9a5a-3bd87355432c_1138x406.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:406,&quot;width&quot;:1138,&quot;resizeWidth&quot;:484,&quot;bytes&quot;:52105,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!uJDE!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5062a702-aee9-442d-9a5a-3bd87355432c_1138x406.png 424w, https://substackcdn.com/image/fetch/$s_!uJDE!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5062a702-aee9-442d-9a5a-3bd87355432c_1138x406.png 848w, https://substackcdn.com/image/fetch/$s_!uJDE!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5062a702-aee9-442d-9a5a-3bd87355432c_1138x406.png 1272w, https://substackcdn.com/image/fetch/$s_!uJDE!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5062a702-aee9-442d-9a5a-3bd87355432c_1138x406.png 1456w" sizes="100vw" loading="lazy"></picture><div></div></div></a><figcaption class="image-caption">Here, the LLM is calling a calculator</figcaption></figure></div><p>LLMs can be taught that certain things are best allocated to another process. This is called tool use by LLMs, and are a way that LLMs can expand their capabilities beyond their limitations. This is not <em><strong>general</strong></em> computation because each of these use cases need to be specifically trained, and the tools themselves are not <em><strong>general</strong></em> reasoning tools.</p><h2>In theory, LLMs can compute, but&#8230;</h2><p>In theory, LLMs alone could do computations when chained, i.e. asking it to carry out each step of the algorithm as a separate computation (with this, you can prove LLMs are Turing Complete). This method of computation is so prohibitively expensive that it&#8217;s practically not done for general computation, but are being explored by researchers in seeing the limits of reasoning by LLMs in solving advanced math, scientific, and even legal / medical problems.</p><h1>AGI? Not yet.</h1><p>In my opinion, Artificial General Intelligence (AGI) requires the ability to do <em><strong>general</strong></em> computation and follow algorithms. The lack of such an ability in the current iteration of the LLM technologies means that they fall short of that definition.</p><p>I think this is an awesome foundation on which we could achieve AGI, and it feels tantalizingly close and achievable. There&#8217;s just a few more large leaps we have to achieve to get there.</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/why-do-llms-work?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Thank you for reading Cal.vin. This post is public so feel free to share it.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/p/why-do-llms-work?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://blog.cal.vin/p/why-do-llms-work?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><h1>LLMs seem magical</h1><p>In some sense, LLMs are magical, and their rapid development has surprised even the most optimistic machine learning researchers. While the future is always difficult to predict, it&#8217;ll definitely be exciting and perhaps we&#8217;ll have a breakthrough that creates Artificial General Intelligence. Until then, LLMs remain only a powerful tool for humans, which very likely will displace some jobs while significantly uplevel-ing the productivity of others.</p><p>I hope you&#8217;ve enjoyed this essay on why LLMs work. Feel free to leave a comment or email me your questions or concerns at <a href="mailto:wave@cal.vin">wave@cal.vin</a>!</p>]]></content:encoded></item><item><title><![CDATA[A penny for my thoughts]]></title><description><![CDATA[If my thoughts are worth a penny, how much should I charge?]]></description><link>https://blog.cal.vin/p/a-penny-for-my-thoughts</link><guid isPermaLink="false">https://blog.cal.vin/p/a-penny-for-my-thoughts</guid><dc:creator><![CDATA[Calvin Li]]></dc:creator><pubDate>Sun, 24 Mar 2024 21:33:04 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!duwf!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>The idiomatic guidance for this is clear: &#8220;a penny for your thoughts&#8221;, so I will only charge a penny for my thoughts (henceforth known as a &#8220;thought-penny&#8221;), however I will adjust for inflation &#128519;. </p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading cal.vin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p><strong>tl;dr:</strong> as of February 24th, 2024, an inflation adjusted thought-penny is &#8776; <strong>$4.14 </strong>&#8230; and with at least 2 posts a month, it would be $8.28 (Substack requires a minimum monthly subscription of $5).</p><h1>Origin</h1><p>The earliest known written record of the idiom is in 1522 from Sir Thomas More&#8217;s book <strong><a href="https://archive.org/stream/fourlastthings00more#page/18/mode/2up">Four Last Things</a></strong> in England.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!duwf!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!duwf!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png 424w, https://substackcdn.com/image/fetch/$s_!duwf!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png 848w, https://substackcdn.com/image/fetch/$s_!duwf!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png 1272w, https://substackcdn.com/image/fetch/$s_!duwf!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!duwf!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png" width="1456" height="1045" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1045,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3649058,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!duwf!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png 424w, https://substackcdn.com/image/fetch/$s_!duwf!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png 848w, https://substackcdn.com/image/fetch/$s_!duwf!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png 1272w, https://substackcdn.com/image/fetch/$s_!duwf!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c54a08b-2040-4b92-b0a9-d5cb673378ca_1766x1268.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">&#8216;As it often happeth that the very face sheweth the mind walking a pilgrimage, in such wise that, not without some note and reproach of such vagrant mind, other folk suddenly say to them, <strong>&#8220;A penny for your thought.&#8221;&#8217; - Sir Thomas More, 1522</strong></figcaption></figure></div><h1>How much is inflation?</h1><p>As Sir Thomas More published in England, let&#8217;s first track the inflation of the British Pound. According to the <strong><a href="https://www.bankofengland.co.uk/monetary-policy/inflation/inflation-calculator">Bank of England inflation calculator</a></strong>, a pound from 1522 is worth &#163;788.45 adjusted for inflation.</p><p>Since I&#8217;m charging USD, &#163;788.45 is equal to $993.27:</p><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!14aS!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F509435fd-8dac-429e-a1db-18a0465b1cd0_1352x634.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!14aS!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F509435fd-8dac-429e-a1db-18a0465b1cd0_1352x634.png 424w, https://substackcdn.com/image/fetch/$s_!14aS!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F509435fd-8dac-429e-a1db-18a0465b1cd0_1352x634.png 848w, https://substackcdn.com/image/fetch/$s_!14aS!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F509435fd-8dac-429e-a1db-18a0465b1cd0_1352x634.png 1272w, https://substackcdn.com/image/fetch/$s_!14aS!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F509435fd-8dac-429e-a1db-18a0465b1cd0_1352x634.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!14aS!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F509435fd-8dac-429e-a1db-18a0465b1cd0_1352x634.png" width="448" height="210.0828402366864" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/509435fd-8dac-429e-a1db-18a0465b1cd0_1352x634.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:false,&quot;imageSize&quot;:&quot;normal&quot;,&quot;height&quot;:634,&quot;width&quot;:1352,&quot;resizeWidth&quot;:448,&quot;bytes&quot;:106333,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!14aS!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F509435fd-8dac-429e-a1db-18a0465b1cd0_1352x634.png 424w, https://substackcdn.com/image/fetch/$s_!14aS!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F509435fd-8dac-429e-a1db-18a0465b1cd0_1352x634.png 848w, https://substackcdn.com/image/fetch/$s_!14aS!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F509435fd-8dac-429e-a1db-18a0465b1cd0_1352x634.png 1272w, https://substackcdn.com/image/fetch/$s_!14aS!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F509435fd-8dac-429e-a1db-18a0465b1cd0_1352x634.png 1456w" sizes="100vw" loading="lazy"></picture><div></div></div></a></figure></div><p>Since there are 100 pennies (a.k.a. pence) (p) to a pound (&#163;), a 1522 penny adjusted for inflation should be $9.93, right? Wrong.</p><h1>Decimalisation: the pre-decimal penny</h1><p>Prior to 1971, there was the <strong><a href="https://en.wikipedia.org/wiki/Penny_(British_pre-decimal_coin)">pre-decimal penny</a></strong> (d) that was worth 1/240th of a pound. Back then, a pound (&#163;) was worth 20 shillings (s), and a shilling (s) was worth 12 pre-decimal pence (d). The penny was withdrawn in 1971, in an event known as &#8220;decimilisation&#8221;, and replaced with the decimal half-penny, with &#189;p worth 1.2d.</p><h1>Final Math</h1><p>A thought-penny is worth $993.27 / 240 = $4.138625 &#8776; <strong>$4.14</strong>.</p><p>Committing to two subscriber posts a month, the final amount is <strong>$8.28</strong> (also because substack requires a minimum of $5 &#9785;).</p><h1>Lock in the rate!</h1><p>I will regularly be updating the new subscription cost to keep up with inflation of the thought-penny. Earlier subscriptions will lock in their previous rates. Assuming inflation stays positive, the cost of delaying subscription to my blog will likely increase over time!</p><p>A more variable factor is the foreign exchange rate between a Pound sterling (&#163;) and a United States Dollar. It would be cheaper to subscribe when the UK economy is weak and the US economy is strong.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading cal.vin! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Testing Labeling and Grouping]]></title><description><![CDATA[This is a label test to see if substack has the desired features for my long-term blog. Please ignore!]]></description><link>https://blog.cal.vin/p/testing-labeling-and-grouping</link><guid isPermaLink="false">https://blog.cal.vin/p/testing-labeling-and-grouping</guid><dc:creator><![CDATA[Calvin Li]]></dc:creator><pubDate>Sun, 24 Mar 2024 20:43:52 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!AglF!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb5ac185f-a363-4144-9bb9-fc5b1511b0e7_144x144.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p></p>]]></content:encoded></item><item><title><![CDATA[Coming soon]]></title><description><![CDATA[This is Calvin.]]></description><link>https://blog.cal.vin/p/coming-soon</link><guid isPermaLink="false">https://blog.cal.vin/p/coming-soon</guid><dc:creator><![CDATA[Calvin Li]]></dc:creator><pubDate>Sun, 24 Mar 2024 20:41:05 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!AglF!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb5ac185f-a363-4144-9bb9-fc5b1511b0e7_144x144.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>This is Calvin.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://blog.cal.vin/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://blog.cal.vin/subscribe?"><span>Subscribe now</span></a></p>]]></content:encoded></item></channel></rss>