Visual Studio 2015 Continuous Integration Tools: Visual Studio Online vs TeamCity

I’ve been reviewing options for a new build controller that supports continuous integration in a Visual Studio 2015 development environment over the last few weeks, and now it’s time to wrap up and compare the two options I tested.

The two tools that I was able to successfully implement were Visual Studio Online and TeamCity 9.1.

Both solutions had their strong and weak points, and which you might choose will depend heavily on your precise needs and development environment.

Why Continuous Integration?

First, let’s examine why you would need a build server and CI in the first place.

Failing early

This is a really important factor when doing agile development. If you have a complex application, it’s inevitable that things will break from time to time. The earlier you know about it, the less effort it will take to fix.

Integration Testing

If your app is hosted in the cloud, you will want to test it in the real world. doing this manually is error prone and time consuming.

Automated deployment

Same as above – deploying complex applications is error prone and time consuming. CI will automate this process

Code Coverage

You can run code coverage tools as part of your build, to make sure that any new code has an associated Unit Test.

Standard Workflow

CI tools can automate more than just the build – once the build is done and tested, the staging server can be backed up, and the new build deployed automatically.

Environment

For the purpose of this comparison, the solution being developed was a web application containing multiple projects, each of which is hosted in it’s own Azure Web App or Web Job. The solution is implemented in Visual Studio 2015, though the front end is a Single Page Application developed in pure HTML/JavaScript.

Setting up the build server

In terms of ease of use, both TeamCity and VSO were very easy to set up and configure, and worked out of the box with Visual Studio 2015. VSO has the edge here, though, as TeamCity requires a dedicated server, whereas VSO is a cloud solution. Another plus for VSO is that it has source control covered, so if you are really starting from scratch, VSO can be a complete solution that works out of the box. With TeamCity, you will need to set up your source control separately, though it supports a wide range of source control systems, and even more with plugins. If you have an existing source control server hosted internally, there’s every chance you can set up TeamCity to use it, whereas with VSO, you will probably end up using VSO’s source control for some part of the process. Git and TFVC are supported in VSO.

Security

VSO uses Microsoft Live accounts and/or Azure subscriptions to manage access to the entire Azure platform, including VSO, Azure hosting and source control. This integrated security can further be managed by syncing with Active Directory. This is a fantastic solution if you are already in an environment where each developer has a windows account and an MSDN subscription.

In contrast, TeamCity has individual user accounts, roles and groups that are specific to TeamCity. This means that managing users in TeamCity becomes an additional administrative task if (when) your team changes.

VSO also has better tools and support for larger development teams, given it’s integrated with the entire Microsoft ecosystem. The downside to VSO is that your source code will be stored on Microsoft’s servers, which can be a security issue in itself.

TeamCity is therefore the better solution if you don’t feel comfortable putting your source code in the cloud.

Features

Both build servers support a huge set of common features related to managing builds, testing and deployment. If your builds are fairly standard, either solution will handle it well. You can set up build steps to call command line tools, manage Nuget packages, run unit tests, deploy to a web server and much, much more.

VSO includes support for a lot more environments, such as Android, iOS and, of course, extremely easy integration with Azure WebApps.

TeamCity can also support the same features, but a lot more work would be required to set up the necessary build agents.

If your needs are those of a typical web application developer, there are enough tools in each solution to handle all your needs. More complex applications are supported by both tools, but will require additional work.

One distinct advantage of VSO is that it comes fully integrated with a suite of tools to support very comprehensive development lifecycle management. This includes source control, defect tracking, QA processes, and project management. TeamCity, in contrast, does one thing very well, and has the advantage if you already have existing processes in place, and just want to add CI.

Usability

There’s not much between the two solutions here. Each one comes with it’s own quirks and perks. Neither product was difficult to use, but I did find that sometimes I spent far too much time looking for a particular option in VSO, mostly because VSO is a much more complex and comprehensive product. In contrast, setting up a project, monitoring builds and viewing statistics and results was dead simple in TeamCity.

It must be said that VSO looks a lot more modern and most of the features have a wow-factor compared to similar tools from other vendors. The Microsoft engineers and designers have done an excellent job to put together such a complex and usable piece of software.

Performance

TeamCity has to get the trophy here, as VSO can be a bit sluggish – both in terms of User Interface, and also the process of initiating a build. Most of this is due to the fact that TeamCity is hosted on the local network, and runs on a pretty decent dedicated server, but the reality is that you are unlikely to match the performance of a locally hosted tool with something that is hosted in the cloud.

Price

Often this is what the choice comes down to. In this case, again, it can be a little confusing in the case of VSO, as price depends on how invested you already are in Microsoft’s products. If you are a MSDN subscriber, you already have a reasonable amount of monthly credit to use Azure services, which includes the cost of making a build in VSO. There are no upfront cost, and instead you are paying for a subscription to a service, so costs will depend on how much you use it.

Overall, considering you don’t need to buy and maintain server infrastructure, the price is quite reasonable, and Microsoft are trying hard to make their own tools as attractive as possible.

Pricing for TeamCity is a lot more like a traditional piece of software. There are 2 versions available: Professional and Enterprise, and the good news is that Professional is completely free.

This version is sufficient for the needs of a small team of developers.

Additional projects and build agents can be added for around $300 each, and the Enterprise version at $1999 removes all restrictions on the number of projects and build agents. However, you will need to include the cost of hosting TeamCity and source control on your own servers, including patching, electricity, maintenance etc. There’s also an overhead for making sure that all the pieces work together, which is not the case in VSO.

Conclusion

It’s difficult to recommend one solution over another – they are both excellent. If you have an existing infrastructure, including servers, source control and a defect tracking system, and you want to continue using these,then TeamCity is probably going to integrate better into your existing infrastructure. On the other hand, if you have none of these things, and want to get up and running quickly, then VSO is going to start looking pretty good.

The only other thing of note is that VSO, being cloud based, will require a decent internet connection. Mostly, this is no longer an issue these days, but if you prefer the assurance that all the parts of your build and deployment process are hosted in-house, then you won’t be disappointed with the features in TeamCity.

Advertisements

Consolidating Package Versions with Visual Studio 2015 Nuget Package Manager

With Visual Studio 2015, there comes version 3 of the Nuget Package Manager .

One of the nice features in it is the ability to consolidate multiple projects to use the same version of a package.

Previously, in a solution that contained multiple related projects, and multiple developers, it was sometimes difficult to ensure that all projects were sharing the same packages. The way to manage this was  (and still can be) to create a powershell script that installs the correct package versions for each project, or to manually install and uninstall packages for each project.

With Nuget Package manager 3.0, there is now a much easier way to maintain the list of packages for a solution. The screenshot below shows the new layout, and the key new feature – the ‘Consolidate’ action.

NPM

Using this action, you can quickly update all the projects in your solution to the same version of a package.

To do this, just right-click on the solution and choose “Manage NuGet Packages for Solution”.

From there, select a package that is installed into multiple projects, and check if the ‘Consolidate’ action is available. If it is, it means that one or more of the projects is using a different version of the package than the selected version. Choosing this action will then display the projects using different versions, as well as the actual version of the package that they are using.

To finish the process, just click the ‘Consolidate’ button.

Repeat this process for every package in the solution, and the mismatched versions are gone.

Automate package management with the package manager console

[UPDATE – 12-Dec-2015]

In case you are not yet using NuGet version 3.3, install it! Consolidation of packages in now a lot easier using the GUI, as there’s a new ‘Consolidate’ button that lists all the packages in the solution that can be consolidated. This is a lot faster than going through one by one.

When you have a large number of projects and packages, it can be much quicker to perform package management functions using the console instead of the user interface. The nuget package manager reference can be found on this page, and I have a listed a few useful commands that I find myself using.

  1. List all packages that have updates available
    get-package -update
  2. Update all packages to the same highest minor version
    foreach ($package in get-package -update) { update-package -Id $package.id -ProjectName $package.projectname -ToHighestMinor -FileConflictAction Overwrite -whatif }
  3. Sync packages in solution to  the same version as the nominated project (Nuget v3 or higher)
    Sync-Package -Id <ProjectName> -WhatIf

The commands that actually make changes to packages.config all have a “-WhatIf” parameter that will show what changes are going to be made without actually making them. It’s important to do the what-if, because running a script on your whole solution can delete or put incorrect references into the packages.config, which may then need to be fixed manually. It’s also wise to use your source control or backup your solution when making changes that potentially affect a large number of files.

Visual Studio 2015 released, Azure MSDN credit increased and new pricing for VM instances

It’s been a busy couple of weeks since the last post.

First up, we were pleasantly surprised that Visual Studio Pro edition with MSDN now gives you $70 in monthly credits. At the same time, the pricing of VM’s was decreased. What this mean is that I can hopefully keep a small VM up and running for more than half the month without running out of credit.

Oh! And Visual Studio 2015 has been released! Aside from a few small issues, existing projects in VS 2013 should work just fine in VS 2015.

The only issues we encountered were related to installation and the Nuget package manager GUI. The latter issue was addressed in a hotfix, and has now been patched properly in an update. For information about the issue, follow this link: https://connect.microsoft.com/VisualStudio/feedback/details/1572078/nuget-crash-in-visual-studio-2015-enterprise

A separate, much more serious issue was found in .NET 4.6, which can result in variables having incorrect values due to an optimization bug in the new .NET compiler (RyuJit). A detailed synopsis of the issue is documented here:

http://blogs.msdn.com/b/dotnet/archive/2015/07/28/ryujit-bug-advisory-in-the-net-framework-4-6.aspx

Overall, the release brings heaps of new features, better support for modern web development, and cross platform support, features I’ll be keen on trying out in the coming weeks.

Building it in the Cloud – Part 3

Introduction

Now that we have a web app in azure, and have connected it to Visual Studio Online in the earlier posts (part 1 and part 2), it’s time to finish up this little project by checking out how the build and deployment pipeline works.

 Continuous Integration

Log into the Visual Studio Online account linked to you Azure subscription, and go to to our project, which should be available from the dashboard. Once you click on the project name, go to the ‘BUILD’ tab.

Build1

You can see the build definition created for you when the web app was linked to Visual Studio Online. You can also go to the ‘CODE’ tab to confirm that all your project files have been committed to Visual Studio Online’s Git repository.

Once that’s confirmed, we need to test the integration with our development environment. For that, let’s go back to Visual Studio 2013, open the project, and make a change. I’ve added an extra value to the Values Controller class.

build2

Now we need to commit our changes to the local repository, and sync to Visual Studio Online.

build3build4

If everything worked correctly, we should shortly see our changes on the production site, and our source code in Visual Studio Online. Let’s log back into Visual Studio Online, and check to see what has occurred after committing our changes.

build5

First, in the ‘CODE’ tab, we can see that our changes were synced to the remote repository – the “value3” change is visible in the code explorer view.

build7

Next, let’s check the ‘BUILD’ tab. Sure enough, there’s a new build there created a minute ago – it has a green tick, so the code was successfully compiled on the hosted build server.

Great, just a couple more things – let’s check the build logs, by right-clicking the build, and selecting ‘Open’.

build9

And finally, we need to open our web site, go to the values API, and confirm that the changes are successfully deployed to production.

build10

 Conclusion

This series of articles has barely scratched the surface of the capabilities in Visual Studio Online. The next step is to schedule automated tests, set up deployment slots, and customize the build process.

However, I hope this brief look at Visual Studio Online’s features is enough to encourage you to look further.

Building it in the Cloud with Visual Studio Online – Part 2

Introduction

Last week, I wrote about some of the reasons why you might want to use Visual Studio Online’s hosted build controller to build and deploy your Azure web apps. For small teams, or individuals, you can get all the benefits of a robust build server and continuous integration for your project, without the hassle of maintenance and administration. This week, I’ll build a simple MVC app, and configure it for source control and continuous integration using Visual Studio Online.

This guide is aimed at people who are somewhat familiar with Visual Studio and the basics of source control, but have not used Visual Studio Online or Team Foundation Server before.

Prerequisites

You will need Visual Studio 2013 (any version), as well as an Azure subscription to complete this walkthrough.

Creating the project

I’m going to switch things around from the usual scenario for Visual Studio Online source control, where you are forced to start with a blank project and open it in Visual Studio – if you need to know how to do that, there are plenty of excellent tutorials, like this one at: http://blogs.msdn.com/

In many scenarios you will have an existing solution that you want to add to a new Visual Studio Online repository, and this is quite simple,  but not well documented.

First, let’s create a basic MVC web app in Visual Studio, and call the app VSOWebDeploy.

NewProjectVS

We will use the Web API template, and configure it to be hosted in on Azure, like so.

SelectTemplate

Once you click OK, you will be prompted to create the Azure Web App. The app name needs to be unique, and you will need to specify a resource group, or create a new one. For the region, choose one that is near to you, and you can leave the database option as ‘No Database’. After clicking ‘OK’ the web app will be created and ready to use.

Once finished, you should be able to run the application with Ctrl-F5, using the local IIS Express server.

Configuring the web app for Visual Studio Online

Now that we have an existing application, let’s create the Visual Studio Online account that will allow us to build the solution online.

First, log into the Azure Management Portal (https://manage.windowsazure.com/), and create a new Visual Studio Online Subscription if yo don’t already have one. Go to “App Services\Visual Studio Online\Quick Create” to do this.

Select the URL (needs to be unique, e.g. <yourname>.visualstudio.com), the Azure Active Directory service and the Region closest to you, and click ‘Create Account’.

Linking the project to Visual Studio Online

For a brand new solution you can just click the ‘Open in Visual Studio’ link, and you’re ready to go. But there’s one extra step to link an existing solution to Visual Studio Online.

Basically, what I will be doing is using a local Git repository for the solution’s source control, and then pushing the master branch up to Visual Studio Online.

Go back into Visual Studio, right click the solution, and choose ‘Add to Source Control’. When prompted, choose Git as the source control provider.Choose Source Control

Next, you will see the Team Explorer – choose to see the ‘Changes’ screen, type a commit message, and click ‘Commit’ to commit the project files to local source control. At this point, your code is being tracked for changes, and the history is stored on your local drive, so let’s go one better and push those changes up to Visual Studio Online.

Click the icon in Team Explorer that looks like a power plug, and either select an empty team project, or choose to create a new team project. If doing the latter, you will be asked to supply a project name – type VSOWebDeploy  and click Create.

In a short while, the new project should be available for use. At this point, you will see a hint in the Team Explorer that you need to clone your repository to get started. This is pretty much what we need, so let’s click that link and clone our repository.

Here, it can get a bit messy, so you need to make sure you do things in the right order, or you’ll get error messages about merge ref not found, which basically means it can’t find anything to merge with.

You can resolve this in a number of ways, and if you are used to command line git, you can just force a push to the remote repository.

A simper way, though, is to go to ‘Unsynced Commits’, enter the remote repository URL, and click ‘Publish’. Your remote repository URL is in the form of https://{Your Account Name}.visualstudio.com/DefaultCollection/_git/{Your Team Project Name>} You can also find the URL by visiting your Visual Studio Online account, and going to the ‘Code’ link, if you have not yet synced to Visual Studio.

Linking the Web App to Visual Studio Online

Next, go to the web apps list, and find VSOWebDeploy in the list of Web Apps. On the dashboard, you will see a link to ‘Set up deployment from source control’.

AzureDashboard1

You will be prompted to choose location of your source code – choose Visual Studio Online, to connect the project to a Visual Studio Online account.

AzureDashboard2

After this, you will be asked to confirm the link, and that’s pretty much it – once linked, you can start using Git and Visual Studio Online builds publish your app directly from Azure, using the Hosted Build Controller.

When working on the code, make sure you remember to commit your changes regularly to the remote repository. By signing in to the Visual Studio Online account, you can explore the code, see a history of the commits, and perform various administrative tasks.

In the next post, I will finish up by setting up an automated build and continuous integration.

Azure Web Apps – Edit in Visual Studio

The ‘Edit in Visual Studio’ feature in the Azure Portal allows you to edit files on the live site, without needing to deploy or publish your app. It also contains some really useful tools for when you need to troubleshoot or configure your web app, and only have access to a web browser.

Follow these steps to enable the feature:

  1. Log into https://manage.windowsazure.com/
  2. Select the Web App, and go to the ‘Configure’ tab
  3. Scroll down to the ‘EDIT IN VISUAL STUDIO’ setting and enable it
  4. Save the changes

Now you can go the DASHBOARD link of the Web App, and find the ‘Edit in Visual Studio Online’ option in the ‘Quick Glance’ section on the right.

Apart from being able to edit files directly on the live site, you can access the following tools:

  • Git – for pushing and pulling to a local repository, or VSO
  • Output Console – for reading debug messages as the site is running
  • Console – a basic shell with some command line tools

The list of command line tools available in the console can be retrieved by typing “help”. Notable ones include the Node Package Manager (npm), Git, Powershell (ps) and msbuild, along with the usual file and directory manipulation tools.

Building it in the Cloud with Visual Studio Online

Introduction

So we recently started a new project at work, hosted on Azure. As part of the initial investigation, I stared poking around in the Azure Portal to see what new toys we might be able to play with.

One of the items under review is our source control, build and deployment tool-chain, which, frankly, is a bit outdated. Since Microsoft recently released an updated Build module for Visual Studio Online, I took the time for some further investigation.

With that in mind, let’s have a look at some of the new features, and how it can be leveraged to improve the build and deployment process.

This week, I’m looking at the Hosted Pool Build Controller

Hosted Build Controller

In the past, we have deployed and maintained our own VM’s for builds and deployment. The advantage of this is that all our source code lives on the internal network, and we are in full control of the entire build process.

Visual Studio Online can provide all that for you in a user friendly portal, where you can define your build process, set up continuous integration or a build schedule, and then receive alerts, or just review logs as needed.

The combination of Hosted Builds, Unit Testing, Web App Deployment Slots and Continuous Integration presents a powerful argument to move your builds into the cloud, but it does come with some risks and downsides.

When would you choose to use Visual Studio Online Hosted Builds?

  • You are already using VSO for source control (TFS or GIT)
  • Your application is fully hosted on Azure
  • Maintaining a separate build server is difficult or too time consuming

Downsides to Hosted  Builds:

  • You are storing your source code in the cloud, which poses a security risk
  • Limited amount of customised build steps are available
  • Support for languages, compilers and source control systems is limited
  • No administrator access to the build controller

In summary, VSO Hosted Builds are a great way to get going quickly, provided you are willing to stay ‘in the box’ with regard to compilers, build and deployment tools. There are also limitations on the size of the application (10Gb as of writing), and the list of installed applications.

The list of features is really great, and growing, so it’s well worth checking it out.

For detailed information about VSO, visit Microsoft’s Visual Studio Online page.

Part 2 of this blog post is a walkthrough on setting up a new web app and using Visual Studio Online for source control.

Part 3 describes how to implement continuous integration in the Visual Studio Online environment.