MSBuild Task Submissions / Improvements

Jul 13, 2011 at 12:35 AM

I've been working on a number of improvments to the MSBuild minification task and wondered if there might be an opportunity to submit some of the improvements back to the community. A few of the changes bring the task much closer to traditional MSBuild task usage, but also introduce some breaking changes.

Specifically:

  • Split the AjaxMin task into two separate tasks - MinifyJavascript and MinifyStylesheet
    • Rationale: The content types are completely different, and it makes sense to specify the MSBuild items distinctly. For example <CssFile Include="**\*.css" />. In addition, the minification settings for each are unique. Not much different than comparing the VB compiler task to the C# compiler task - similar output and properties, but not the same.
  • Rename the property for the input files to "SourceFiles"
    • Rationale: Matches existing convention used in MSBuild (ie/ CopyTask)
  • Create a new, optional, "DestinationFiles" property.
    • Rationale: If minification to a new file is required (ie/ .min.js) an MSBuild item can be specified. MSBuild item transformations can be used to generate the ".min." automatically, without code inside the task.
  • Create a new "MinifiedFiles" output property containing the files (and metadata) for all of the minified output files.
  • Remove JsSourceExtensionPattern and JsTargetExtension.
    • Rationale: Doesn't follow correct MSBuild patterns. MSBuild item transformations should be used to do this. (ie/ new DestinationFiles property). Conflicting feature, and makes it difficult/confusing to compact to the same file.
  • Replace "catch (Exception)"
    • Rationale: Discouraged coding practice. MSBuild tasks do not catch global exceptions internally, since MSBuild will automatically handle them gracefully. Replace with targeted exception handling where necessary - catch (IOException)

The existing MSBuild task can be left for backwards compatibility, while the new ones would fit closer to traditional MSBuild syntax. Let me know if any of this sounds useful at all. :)

Coordinator
Jul 13, 2011 at 12:50 AM

I'm very interested in the pain-points for the build task -- I didn't write it (the ASP.NET folks did), and I'm not particulary a user of the task, so I'm a little behind the curve when it comes to this topic. I'll have to keep the existing task so we won't break the ASP.NET folks, but I'm all for adding new ones to make it simpler going forward. The current task seems a little unweildy to me, if I'm totally honest. I agree that there should be two different tasks; I can't answer why it was written as a single task (it actually goes through some interesting gyrations in property naming in order to get it to work properly). You mention correct MSBuild patterns -- can you point me to a webpage that describes those? I'd love to educate myself on such patterns.

Jul 13, 2011 at 4:03 PM

"Patterns" probably wasn't the right word, sorry. I haven't read any official documents - just information that I gleaned from observing the core built-in MSBuild tasks and a lot of pain writing our custom build environment :)

The book "Using MSBuild and Team Foundation Build" by Sayed Hashimi and William Bartholomew was definitely a help.

What I'll do is pull down the latest sources and try to match the existing development style as I split out the two tasks. At very least the code can be used as a discussion point to see if there's interest in heading in that direction. :)

Jul 13, 2011 at 4:40 PM

One quick question - should I use "JScript" instead of "Javascript" when naming classes and class members?

If I remember right, Microsoft has legal restrictions against the use of the word "Java" in their products due to the settlement with Sun years ago. Not sure if it applies to ajaxmin or not.

Coordinator
Jul 13, 2011 at 4:48 PM

Thanks for the book pointer -- I'll have to pick up a copy. And I wouldn't worry about that level of naming detail. I can't actually accept code submissions from non-Microsoft employees anyway, so this is merely a discussion of your pain-points regarding the current MS Build task supplied with AjaxMin, and your suggestions for how a proper implementation of a more-effective MSBuild interface might look.

Jul 14, 2011 at 5:03 PM

Ahhh right, I always forget about that :)

I'm about 90% done - I'll finish up my new build task today and run a few more tests with how it integrates into our build environment. Once that's done I'll post back here with some more detailed ideas and discussion.

The internal architecture of the minifier got me thinking - it's basically a compiler re-emits code rather than instructions. It might be interesting to create a true "JavascriptCompile" MSBuild task that takes multiple JS files as input, fuses them together, then runs through the normal minification process. I don't know too much about the internals, but depending on how it builds its in-memory representation of the code, it may automatically handle issues like dependencies between files and emit the new code correctly regardless of the order the source files are fused together.

For example, internally here we have a javascript component that we distribute out to our clients to integrate into their e-commerce sites. It consists of a few modules broken up into different javascript code files - mainly to make the code easier to maintain and allow the team to work on different parts of it collaboratively at the same time (full change control tracking and versioning through TFS of each separate file/component). Right now we run a fairly simple custom MSBuild task that concatenates the files in a specific order before being minified.

Given the direciton of HTML5, IE10, and the web in general - I can't help but wonder if our approach has some larger merit. Consider a "Javascript Project" in Visual Studio that works with multiple JS source files and compiles out to a single "library" - or minified JS file. The tricky issue of course, is how fine-grained people might like to slice their JS files. Is it preferential to have one large "app" javascript file, or many small pieces as in traditional web development? A lot of game developers are starting to target Javascript as a platform, and in that case a single script (or a small number of larger scripts) would probably work better and fit nicely into the model I just proposed. Not unlike VS2010's dbproj - you write the SQL as code, and the compiler slices, dices, validates, and can emit a single script.

Random brainstorming aside, I'll post back with the minification task ideas soon :)

Coordinator
Jul 14, 2011 at 5:34 PM

We have something similar here at MSN. I mentioned before I'm not a user of the AjaxMin build task; we don't use it at all here -- we have many, many small-to-medium sized JS chunks that we concatenate together based on which features are being used on which pages, then minify the results into large JS files that require only a single download from the client. For performance reasons, it's a terrible design to have a lot of different JS files linked to from your web page. The fewer HTTP round-trips, the better.

Jul 14, 2011 at 10:54 PM
Edited Jul 15, 2011 at 12:09 AM

Here are some thoughts on build task design:

Two tasks, potentially named "MinifyJavascript" and "MinifyStylesheet". A common abstract base-class could potentially be used to allow for for code-reuse.

Implementation Notes:

  • Avoid mapping MSBuild properties directly to properties in the minfication classes. In general, MSBuild properties should not throw exceptions (ie/ argument exceptions) when an invalid valid is set. Property verification occurs at the start of Execute and logs task errors, rather than throwing exceptions.
  • As mentioned above, validate the input of MSBuild properties at the start of Execute. If a property is invalid log a message to the MSBuild error log and return false. Don't throw exceptions in the property setters if it can be avoided.
  • Return a list of items that were modified by the task. In this case, a read-only ITaskItem[] output property containing a list of all of the minified files that were written. The information can be used to create new MSBuild items and passed into other tasks (generally all of the built-in MSBuild tasks have an output property or two).
  • List data (with the exception of "file" lists) should be a string. At a minimum it should take a "trimmed" semi-colon delimited string - if an item list is specified for the property MSBuild will automatically turn it into a semi-colon delimited string. Most of the built-in MSBuild tasks support space, comma, and semi-colon as list delimiters.
    For example: value.Split(new char[] { ',', ';', ' ' }, StringSplitOptions.RemoveEmptyEntries); The pattern is user-friendly since it automatically trims the list and handles extra/leftover spaces between delimited items.
  • The Copy task populates its output list property with the item metadata from the SourceFiles items. It may be worth considering doing the same:
    For example: SourceFiles[i].CopyMetadataTo(DestinationFiles[i]);  outputFiles.Add(DestinationFiles[i]);  this.MinifiedFiles = outputFiles.ToArray();
  • In my own implementation I pass "null" to the subcategory parameter of Log.LogError and Log.LogWarning (see: LogContextError). The MSBuild compiler tasks leave this blank, and the information is generally not useful. (ie/ "run-time"). Consistent formatting with the other compiler tasks is probably better than continuing to include it. 
  • Recommend taking out the "skip target file if it's readonly" behavior. It's inconsistent with MSBuild task behavior (from what I can tell) - instead, log an error but continue processing the next item in the SourceFiles list regardless.
  • If a source file doesn't exist, treat it the same as any other IOException thrown from File.ReadAllText. (Ie/ log an error "Unable to minify "source file" to "destination file".")
     

Common Properties:

  • ITaskItem[] SourceFiles
    Specifies the source files to minify. [Required]

  • ITaskItem[] DestinationFiles
    Specifies the list of files to minify the source files to.
    This list is a one-to-one mapping with the list specified in the SourceFiles parameter.
    ie/ the first entry specified in SourceFiles will be minified to the first entry specified in DestinationFiles, and so on.
    If DestinationFiles is blank, the source files will be overwritten with the minified code.

    (Although a blank DestinationFiles would cause in-place minification of the source files and could overwrite the original code, in a build environment this may be the main use-case - for example, in-place minification of all Javascript and CSS resources)

  • ITaskItem[] MinifiedFiles
    Contains the items that were successfully minified. [Output]. "Get" property only, no public setter.

  • int WarningLevel
    Specifies the warning level for the task to display

  • bool TreatWarningsAsErrors
    Specifies whether all warnings are treated as errors

  • string WarningsNotAsErrors
    Delimited list (comma, space, semi-colon) of integer error codes that should still be treated as warnings if "TreatWarningsAsErrors" is true

  • string DisabledWarnings
    Delimited list (comma, space, semi-colon) of integer error codes that should be hidden, rather than treated as warnings. Ie/ to suppress JS0012 and JS0013, you would specify "12,13" or "12;13"

  • string OutputMode
    "SingleLine" or "MultipleLines". Specifies how minified code should be formatted.
    Note that the "enum" is inconsistent within the minifier - the CSS minifier uses a different enum/values than the JS minifier. This property would be standardized and common for both, regardless of the internal implementation.

  • int IndentSize
    Specifies the number of spaces per indent level when OutputMode is set to MultipleLines

  • bool AllowAspNetCodeBlocks
    Specifies whether embedded asp.net code blocks should be allowed.

Javascript Minifier ("MinifyJavascript" task)

  • string DefineConstants
    Delimited list (comma, space, semi-colon) of preprocessor symbols
  • string KnownGlobals
    Delimited list (comma, space, semi-colon) of known global variables
  • string Comments
    None or Important. Specifies which types of comments should be preserved in the output.
  • bool RemoveUnusedCode
    Specifies whether unused code, such as uncalled local functions, should be removed.
  • bool RemoveDebugStatements
    Specifies whether debug statements should be removed.
  • string DebugNamespaces
    Delimited list (comma, space, semi-colon) of the namespaces that should be removed when RemoveDebugStatements is true
  • bool PreserveFunctionNames
    Specifies whether all function names must be preserved and remain as-named
  • string PreserveLocalNames
    None, Localized, or All. Specifies whether local variables and function names must be preserved and remain as-named. The default is None.
  • string EvalTreatment
    Ignore, MakeImmediateSafe, or MakeAllSafe. Specifies how variables and functions scoped around eval statements should be treated. The default is Ignore.

Task properties no longer included:

  • JsTargetExtension, CssTargetExtension, JsSourceExtensionPattern, CssSourceExtensionPattern
    Handled by SourceFiles and DestinationFiles properties
  • JsIgnoreConditionalCompilation
    The default behavior likely works for all users. Conditional compilation comments are part of the behavior of the script and were placed there intentionally. It's extremely unlikely similar comments were accidentally added. Removed property to simplify surface area of task.
  • JsMacSafariQuirks
    Unlikely many people would want to save a few characters at the risk of breaking a major browser. Very minor/unlikely use-case for the build task.
  • JsMinifyCode
    Not required - the task should always minify code. MSBuild tasks do not seem to have a "validation only" modes, from what I can tell.

    I excluded other properties as well that I thought only served to change the appearance of the code, not work around special cases or usage patterns. If every option was added to the task, it would become a bit more difficult to use and find the properties that matter the most.

I didn't think through the last few properties that would need to be added to the "StylesheetMinifier" task, but I'm assuming they would be fairly straightforward.

Usage Patterns:

To easily minify to a .min.js file, MSBuild transformations can be used, like so:

  <Target Name="MinifyScripts">
    <ItemGroup>
      <JavaScriptFile Include="**\*.js" />
    </ItemGroup>

    <MinifyJavaScript SourceFiles="@(JavaScriptFile)" 
                      DestinationFiles="@(JavaScriptFile->'%(RelativeDir)%(Filename).min%(Extension)')" />
  </Target>

In this example, @(JavaScriptFile) would include all .js files in the MSBuild script directory and all subdirectories. The transformation @(JavaScriptFile->'') creates a new list with the same number of entries as the original @(JavaScriptFile) list. %(RelativeDir) %(Filename) %(Extension) are examples of "well-known" metadata that exists for each file item in the list.

To hard-code unique destination filenames, two lists can be used:

  <Target Name="MinifyScripts">
    <ItemGroup>
      <JavaScriptFile Include="Sample1.js" />
      <JavaScriptFile Include="Sample2.js" />
      
      <MinifiedJavaScriptFile Include="MyCompressedSample1.js" />
      <MinifiedJavaScriptFile Include="SecondSample.minified.js" />
    </ItemGroup>

    <MinifyJavaScript SourceFiles="@(JavaScriptFile)" 
                      DestinationFiles="@(MinifiedJavaScriptFile)" />
  </Target>

To minify a single file, a list can be used (as above) or directly specified:

<MinifyJavaScript SourceFiles="Sample1.js" DestinationFiles="Sample1.min.js" />

Properties that take delimited lists (ie/ "Constant1,Constant2") can also be supplied using either items or directly via the property:

    <ItemGroup>
      <JavaScriptGlobal Include="jQuery" />
      <JavaScriptGlobal Include="$" />
    </ItemGroup>
    
    <MinifyJavaScript SourceFiles="@(JavaScriptFile)" 
                      DestinationFiles="@(JavaScriptFile->'%(RelativeDir)%(Filename).min.%')" 
                      KnownGlobals="@(JavaScriptGlobal)" />
or
    <MinifyJavaScript SourceFiles="@(JavaScriptFile)" 
                      DestinationFiles="@(JavaScriptFile->'%(RelativeDir)%(Filename).min.%')" 
                      KnownGlobals="jQuery,$" />

Files can be minified in place for release builds. This simplifies the development workflow and doesn't require any changes to <script src=""> paths between debug and release builds. To minify "in-place" leave off DestinationFiles.

  <Target Name="MinifyJavaScript" Condition="'$(Configuration)' == 'Release'">
    <ItemGroup>
      <JavaScriptFile Include="**\*.js" />
    </ItemGroup>
    
    <MinifyJavaScript SourceFiles="@(JavaScriptFile)" />
  </Target>