{"id":101,"date":"2016-03-29T11:23:19","date_gmt":"2016-03-29T15:23:19","guid":{"rendered":"http:\/\/www.schveiguy.com\/blog\/?p=101"},"modified":"2021-10-20T12:46:30","modified_gmt":"2021-10-20T16:46:30","slug":"import-changes-in-d-2-071","status":"publish","type":"post","link":"https:\/\/www.schveiguy.com\/blog\/2016\/03\/import-changes-in-d-2-071\/","title":{"rendered":"Import Changes in D 2.071 [Updated]"},"content":{"rendered":"<p><em>Note: This post has been updated on 8\/29\/2016 with new information on mixin template imports.<\/em><\/p>\n<p>In the upcoming version of D, several changes have been made to the import system, including fixes for 2 of the oldest bugs in D history.<\/p>\n<p>There&#8217;s bound to be a lot of confusion on this, so I wrote this to try and explain the rules, and the reasoning behind some of the changes. I&#8217;ll also explain how you can mitigate any issues you have in your code base.<\/p>\n<h2>Bugs 313 and 314<\/h2>\n<p>Links: <a href=\"https:\/\/issues.dlang.org\/show_bug.cgi?id=313\">313<\/a> and <a href=\"https:\/\/issues.dlang.org\/show_bug.cgi?id=314\">314<\/a><\/p>\n<h3>Description<\/h3>\n<p>Private imports are not supposed to infiltrate the modules they are imported in. If you import <code>a<\/code>, and <code>a<\/code> imports <code>b<\/code> privately, then you should not have any access to <code>b<\/code>&#8216;s symbols. However, before this was fixed, you could access <code>b<\/code> symbols via the <em>Fully Qualified Name<\/em>. A FQN is where you list all packages, including subpackages, separated by dots, to access a symbol. For example <code>std.stdio.writeln<\/code>.<\/p>\n<p>In addition, when importing a module using <em>static<\/em>, <em>renamed<\/em>, or <em>selective<\/em> imports, the imported symbols were incorrectly made public to importing modules.<\/p>\n<p>An example:<\/p>\n<div class=\"oembed-gist\"><script src=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a.js?file=ex1_a.d\"><\/script><noscript>View the code on <a href=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a\">Gist<\/a>.<\/noscript><\/div>\n<div class=\"oembed-gist\"><script src=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a.js?file=ex1_b.d\"><\/script><noscript>View the code on <a href=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a\">Gist<\/a>.<\/noscript><\/div>\n<div class=\"oembed-gist\"><script src=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a.js?file=ex1_main.d\"><\/script><noscript>View the code on <a href=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a\">Gist<\/a>.<\/noscript><\/div>\n<p>With 2.070 and prior versions, compiling this works just fine. With 2.071 and above, you will get either a deprecation warning, or an error.<\/p>\n<p>Note that the <code>private<\/code> qualifier is only for illustration. This is the default import protection for any imports.<\/p>\n<p>For an example of how selective imports add public symbols:<\/p>\n<div class=\"oembed-gist\"><script src=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a.js?file=ex2_a.d\"><\/script><noscript>View the code on <a href=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a\">Gist<\/a>.<\/noscript><\/div>\n<div class=\"oembed-gist\"><script src=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a.js?file=ex2_main.d\"><\/script><noscript>View the code on <a href=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a\">Gist<\/a>.<\/noscript><\/div>\n<p>With 2.070, this compiled just fine. However, <code>printf<\/code> is supposed to be a private symbol of module <code>ex2_a<\/code>. With 2.071 and above, this will trigger a deprecation warning. In the future, the code will trigger an error.<\/p>\n<h3>Selective imports and FQN<\/h3>\n<p>A combination of both 313 and 314 is when you use a selective import, and expect the Fully Qualified Name to also be imported. This is not what the selective import was supposed to do, it was only supposed to add the symbols requested.<\/p>\n<p>An example:<\/p>\n<div class=\"oembed-gist\"><script src=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a.js?file=ex3_main.d\"><\/script><noscript>View the code on <a href=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a\">Gist<\/a>.<\/noscript><\/div>\n<p>In this example, <code>std.stdio.writeln<\/code> is not actually supposed to be imported, only <code>write<\/code> is supposed to be imported (and even the FQN <code>std.stdio.write<\/code> isn&#8217;t imported!). We have to import <code>std.range<\/code>, because otherwise this would not compile (ironically, the package <code>std<\/code> is not imported by the selective import unless there is another import of the FQN).<\/p>\n<p>In 2.070, this produces no warning or error. In 2.071 and beyond, this will produce a deprecation warning, and eventually an error.<\/p>\n<h3>Fixing problematic code<\/h3>\n<p>In order to fix such code, you have to decide what was intended. If your code really was supposed to publicly import the symbols, prepend <code>public<\/code> to the import statement. This brings all the symbols imported into the namespace of the module, so any importing module also sees those symbols. In our example 2 above, this would mean adding <code>public<\/code> to the import statement in ex2_a.d<\/p>\n<p>If the imported module was <em>not<\/em> supposed to publicly expose the symbols, then you need to fix <em>all importing modules <\/em>with this problem. In our example, this would mean adding <code>import core.stdc.stdio;<\/code> to the top of ex2_main.d.<\/p>\n<p>In the case of accidentally exposing the FQN of symbols that were privately imported, this is typically an issue with the importing module, not the imported one. In this case, you need to add an import. In our example 1 case, this would mean adding an import for <code>ex1_a<\/code> module to ex1_main.d.<\/p>\n<p>For example 3, you can achieve the original behavior by both selectively importing the symbol, and statically importing the module. Just add <code>static import std.stdio;<\/code> to your scoped imports. Alternatively, you can add <code>writeln<\/code> to the selectively imported symbols, and use the unqualified name instead of the FQN.<\/p>\n<p>For an example of how Phobos was fixed for this problem (there were thousands of messages in every build with deprecation warnings), see the <a href=\"https:\/\/github.com\/D-Programming-Language\/phobos\/pull\/4016\">PR I created.<\/a><\/p>\n<h2>Bug 10378<\/h2>\n<p>Links: <a href=\"https:\/\/issues.dlang.org\/show_bug.cgi?id=10378\">10378<\/a> <a href=\"https:\/\/github.com\/D-Programming-Language\/dmd\/pull\/5445\">Pull Request<\/a><\/p>\n<h3>Description<\/h3>\n<p>Another import-related bug fix is to prevent unintentional hijacking of symbols inside a scoped import. Such imports are not at module level, and import inside a function or other scope. These imports are only valid within the scope. Prior to 2.071, such imports were equivalent to importing every symbol in the imported module <em>into the namespace<\/em> of that scope. This overrode any other symbol in that namespace, including local variables in outer scopes. An example:<\/p>\n<div class=\"oembed-gist\"><script src=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a.js?file=ex4_a.d\"><\/script><noscript>View the code on <a href=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a\">Gist<\/a>.<\/noscript><\/div>\n<div class=\"oembed-gist\"><script src=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a.js?file=ex4_main.d\"><\/script><noscript>View the code on <a href=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a\">Gist<\/a>.<\/noscript><\/div>\n<p>In 2.070 and prior, the assert above used <code>ex4_a<\/code>&#8216;s definition of <code>foo<\/code>, not the local variable. In 2.071 and beyond, the local <code>foo<\/code> has precedence. The precedence rules work like this:<\/p>\n<ol>\n<li>Any <strong>local symbols<\/strong> are examined first. This includes selective imports which are aliased into the local scope.<\/li>\n<li>Any <strong>module-level<\/strong> symbols are examined.<\/li>\n<li>Any symbols <strong>imported<\/strong> are examined, starting with the <strong>most derived scope<\/strong> imports, all the way to module-level imports.<\/li>\n<\/ol>\n<p>Note that this may be a <strong>breaking change<\/strong>, as demonstrated by the example.<\/p>\n<h3>Why did we change this?<\/h3>\n<p>This was changed because any symbol added to an import can drastically affect any code that uses non-selective scoped imports, hijacking the symbol in ways that the author cannot predict. While there is still potential for hijacking, since scoped imports override any module-level or higher level scoped imports, at least symbols that are locally defined are not affected. These are the symbols under direct control of the author of the module, and they should always have precedence.<\/p>\n<p>A common change to a module is to move imports inside the scope of functions or types that are the only users of that import. This helps avoid namespace pollution. However, given that local module functions had precedence over imported ones, but scoped imports would take precedence away, this move was not always what the user intended. For this reason, module functions now <em>always<\/em> have precedence over non-selective scoped imports.<\/p>\n<h3>Fixing problematic code<\/h3>\n<p>This one is a little more nuanced. It may be that you wished to override the local symbols! In this case, use a selective import. Selective imports alias the symbols selected into the local scope, overriding any other symbols defined at that point. In our example, if we expected <code>foo<\/code> to refer to <code>ex4_a.foo<\/code>, then we would use an import like this: <code>import ex4_a: foo;<\/code><\/p>\n<p>In addition, you can use the FQN instead of using the simple name. I would recommend using a <em>static<\/em> or <em>renamed<\/em> import in that case.<\/p>\n<h2>Imports from mixin templates ((Thanks to <a href=\"http:\/\/www.schveiguy.com\/blog\/2016\/03\/import-changes-in-d-2-071\/#comment-45\">captaindet<\/a> for bringing this issue to my attention))<\/h2>\n<p>Links: <a href=\"https:\/\/forum.dlang.org\/post\/nl4vse$egk$1@digitalmars.com\">Forum discussion<\/a>, <a href=\"https:\/\/issues.dlang.org\/show_bug.cgi?id=15925\">issue 15925<\/a><\/p>\n<h3>Description<\/h3>\n<p>A somewhat controversial change with 2.071 is the effect mixins can have with\u00a0imports. If you have a mixin template which imports a module, then use that template within a class or struct, the import is only considered while inside the mixin template. It is not considered when inside the class or struct. For example:<\/p>\n<div class=\"oembed-gist\"><script src=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a.js?file=ex5_a.d\"><\/script><noscript>View the code on <a href=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a\">Gist<\/a>.<\/noscript><\/div>\n<div class=\"oembed-gist\"><script src=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a.js?file=ex5_main.d\"><\/script><noscript>View the code on <a href=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a\">Gist<\/a>.<\/noscript><\/div>\n<p>The previous version would allow the import to be considered where the mixin occurs. In order to have a mixin template add an imported symbol, you can selectively import the symbol. In this case, static import will not work:<\/p>\n<div class=\"oembed-gist\"><script src=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a.js?file=ex5_a2.d\"><\/script><noscript>View the code on <a href=\"https:\/\/gist.github.com\/schveiguy\/bfd3048f7f90353ba04a\">Gist<\/a>.<\/noscript><\/div>\n<h3>Why did we change\u00a0this?<\/h3>\n<p>The explanation <a href=\"https:\/\/issues.dlang.org\/show_bug.cgi?id=15925#c18\">seems to be<\/a> that allowing such imports can create a form of hijacking. Since a class-level import would override a module-level import, a user may not realize that the mixed-in import is present, and therefore overriding a module-level import that is in the local module. The hijacking can come after the fact, in the imported module, without the user&#8217;s knowledge or any changes in his code.<\/p>\n<h3>Fixing problematic code<\/h3>\n<p>There isn&#8217;t a very easy\u00a0way to rectify this problem. The only solution is to selectively import all the symbols you may need from that other module within the mixin.<\/p>\n<h2>Transitional Switches<\/h2>\n<p>The new version of the compiler comes with two new transitional switches that you can use to find or ignore these errors (note that these affect the mixin template imports as well):<\/p>\n<p style=\"padding-left: 30px;\"><strong>-transition=checkimports<\/strong>: This switch will warn you if you have code that behaved differently prior to issue 10378 being fixed. Note that this may slow down compilation notably, hence it&#8217;s not the default ((Thanks to <a href=\"https:\/\/forum.dlang.org\/post\/ecythjoqogflfygsjggh@forum.dlang.org\">Dicebot<\/a> for pointing this out))<\/p>\n<p style=\"padding-left: 30px;\"><strong>-transition=import<\/strong>: This switch reverts behavior back to the import rules prior to 10378 being fixed. <em>Only use this switch as a stop-gap measure until you can fix the code!<\/em><\/p>\n<h2>General Recommendations<\/h2>\n<p>Because importing external modules that are outside your control can lead to hijacking, I recommend never importing a module at a scoped level that isn&#8217;t selective, static, or renamed. This gives you full control over what invades your namespace. The compiler will protect you now a little bit better, but it&#8217;s always better to defend against namespace pollution from an uncontrolled module.<\/p>\n<h2>References<\/h2>\n<p><a href=\"http:\/\/dlang.org\">D programming language<\/a><\/p>\n<p><a href=\"http:\/\/dlang.org\/spec\/module.html#import-declaration\">D Import Spec<\/a><\/p>\n<p><a href=\"https:\/\/issues.dlang.org\/show_bug.cgi?id=313\">Issue 313<\/a><\/p>\n<p><a href=\"https:\/\/issues.dlang.org\/show_bug.cgi?id=314\">Issue 314<\/a><\/p>\n<p><a href=\"https:\/\/issues.dlang.org\/show_bug.cgi?id=10378\">Issue 10378<\/a><\/p>\n<p><a href=\"https:\/\/issues.dlang.org\/show_bug.cgi?id=15925\">issue 15925<\/a><\/p>\n<p><a href=\"http:\/\/dlang.org\/download.html\">D Compiler Download<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Note: This post has been updated on 8\/29\/2016 with new information on mixin template imports. In the upcoming version of D, several changes have been made to the import system, including fixes for 2 of the oldest bugs in D history. There&#8217;s bound to be a lot of confusion on this, so I wrote this [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,2],"tags":[],"class_list":["post-101","post","type-post","status-publish","format-standard","hentry","category-dlang","category-prog"],"_links":{"self":[{"href":"https:\/\/www.schveiguy.com\/blog\/wp-json\/wp\/v2\/posts\/101"}],"collection":[{"href":"https:\/\/www.schveiguy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.schveiguy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.schveiguy.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.schveiguy.com\/blog\/wp-json\/wp\/v2\/comments?post=101"}],"version-history":[{"count":9,"href":"https:\/\/www.schveiguy.com\/blog\/wp-json\/wp\/v2\/posts\/101\/revisions"}],"predecessor-version":[{"id":159,"href":"https:\/\/www.schveiguy.com\/blog\/wp-json\/wp\/v2\/posts\/101\/revisions\/159"}],"wp:attachment":[{"href":"https:\/\/www.schveiguy.com\/blog\/wp-json\/wp\/v2\/media?parent=101"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.schveiguy.com\/blog\/wp-json\/wp\/v2\/categories?post=101"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.schveiguy.com\/blog\/wp-json\/wp\/v2\/tags?post=101"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}