Using Kudu to Access Azure Website Extensions

There’s a little-known but excellent feature of Azure web apps that makes it easy to access the debugging and development extensions of your Azure website. If you navigate directly to https://your_site.scm.azurewebsites.net/ (replace “your_site” with the real name of your web site) you will find Microsoft’s Kudu service – the engine behind git deployments for Azure web sites.

There’s tons of useful  information you can see using this tool, such as process information, environment variables, server variables and a lot more.

More importantly, it’s also a portal to easily install and access site extensions, such as Visual Studio Online (which you can use to edit files in place on your web site).

For example, to access Visual Studio Online, select the ‘Site Extensions’ menu. There you will find listed all the available site extensions. The first should be VSO. You will need to enable it first, which requires a website restart, but once done, you can access VSO anytime using the following URL format:

https://your_site.scm.azurewebsites.net/dev/

There are a lot more tools available through Kudu, and exhaustive documentation is available on their wiki: https://github.com/projectkudu/kudu/wiki/Azure-Site-Extensions

Kudu is a fantastic tool for development and debugging of your Azure web sites.

For everyday management and monitoring of a web site, the new Azure Portal is a great tool, but to easily access tools and information that are useful to a development team, Kudu is far more powerful.

Duplicating an Azure database with row level security to a new resource group

Introduction

One of the benefits of using Azure resource groups is being able to quickly create identical environments using resource group templates and powershell scripts, so when I was asked to quickly whip up a new test environment, I was prepared, and feeling pretty good about having automated all the things in advance.

Using the template I had created earlier, I had a shiny new environment within a few minutes. Then I thought.. but what about the data?

The Problem

The solution we are developing uses a Sql server row level security policy to provide access restrictions, and one of the drawbacks of this method is that it’s not easy to extract data using traditional data extraction tools, as every login only sees their own data (as it should be). We could have dropped the policy, created a backup (bacpac) file, and then imported to the new empty database. However, this would require a little downtime to prevent users seeing each other’s data, and anyway, that’s not really an elegant solution.

What I wanted to achieve was:

  • Make a copy of an existing database, including all data, users, and schema information.
  • Keep the same SID (security identifiers) on the new server, so that the row level security wasn’t affected.
  • Avoid modifying the original database in any way.
  • Create the new database in a different resource group.
  • Provide a solution that could be automated.

The Solution

The first step was to create a new database, and populate it with the data from the existing one. The easiest way is to use the portal to create the new database, and specify a backup source, which should list your existing database. Once you click  ‘Create’, Azure will do the hard work to import all the data, users and schema information for you.

Step 1. Select source backup.

Add_database

Step 2.
The next bit is to recreate the logins using the same SID’s on the new server. This is important as row level security uses the SID to determine which user owns which data. To help in doing this, I wrote a small Sql script that needs to be run on the source database. It prints a Sql statement that creates the logins, and maps the logins to the users in the new database.


declare @sid varchar(200)
declare @name varchar(200)
declare @sql varchar(4000)
set @sql = ''

DECLARE my_cursor CURSOR FOR
select name, dbo.fn_varbintohexstr(sid) from sys.sql_logins
OPEN my_cursor
FETCH NEXT FROM my_cursor INTO @name, @sid
WHILE (@@fetch_status <> -1)
BEGIN

set @sql = @sql + 'CREATE LOGIN '+ @name +' WITH PASSWORD = '''', SID = ' + @sid + CHAR(13)+CHAR(10)
set @sql = @sql + 'ALTER USER '+ @name +' WITH LOGIN = ' + @name +CHAR(10)
FETCH NEXT FROM my_cursor INTO @name, @sid
END

CLOSE my_cursor
DEALLOCATE my_cursor

print @sql

Step 3.

Copy the output of the previous step, log into the new server as admin, connect to the new database, and run the generated script. The script will create logins on the new server, and then link the login to the user. They key ingredient is to create the login with the correct SID. For this scenario, the password is not important, but if you need to also get the password from the old server, you will need to modify the script to create a login with a given password hash.

Step 4.
Publish the application.
With the hard work done, the last step is to just publish the code. To do this, you can download the publish settings of your application from the portal, import them into Visual Studio, and publish.

Final Thoughts

This solution at this stage was a bit of a quick hack – the next obvious step is to wrap this all into a nice package for easy re-usability. With this process, we are now able to take a snapshot of the current state of our solution, and replicate the entire environment, including data and security, in a little over 30 minutes. While this solution was implemented and tested on Azure, the same process should work in SQL Server 2016.

Visual Studio 2015 released, Azure MSDN credit increased and new pricing for VM instances

It’s been a busy couple of weeks since the last post.

First up, we were pleasantly surprised that Visual Studio Pro edition with MSDN now gives you $70 in monthly credits. At the same time, the pricing of VM’s was decreased. What this mean is that I can hopefully keep a small VM up and running for more than half the month without running out of credit.

Oh! And Visual Studio 2015 has been released! Aside from a few small issues, existing projects in VS 2013 should work just fine in VS 2015.

The only issues we encountered were related to installation and the Nuget package manager GUI. The latter issue was addressed in a hotfix, and has now been patched properly in an update. For information about the issue, follow this link: https://connect.microsoft.com/VisualStudio/feedback/details/1572078/nuget-crash-in-visual-studio-2015-enterprise

A separate, much more serious issue was found in .NET 4.6, which can result in variables having incorrect values due to an optimization bug in the new .NET compiler (RyuJit). A detailed synopsis of the issue is documented here:

http://blogs.msdn.com/b/dotnet/archive/2015/07/28/ryujit-bug-advisory-in-the-net-framework-4-6.aspx

Overall, the release brings heaps of new features, better support for modern web development, and cross platform support, features I’ll be keen on trying out in the coming weeks.

Building it in the Cloud – Part 3

Introduction

Now that we have a web app in azure, and have connected it to Visual Studio Online in the earlier posts (part 1 and part 2), it’s time to finish up this little project by checking out how the build and deployment pipeline works.

 Continuous Integration

Log into the Visual Studio Online account linked to you Azure subscription, and go to to our project, which should be available from the dashboard. Once you click on the project name, go to the ‘BUILD’ tab.

Build1

You can see the build definition created for you when the web app was linked to Visual Studio Online. You can also go to the ‘CODE’ tab to confirm that all your project files have been committed to Visual Studio Online’s Git repository.

Once that’s confirmed, we need to test the integration with our development environment. For that, let’s go back to Visual Studio 2013, open the project, and make a change. I’ve added an extra value to the Values Controller class.

build2

Now we need to commit our changes to the local repository, and sync to Visual Studio Online.

build3build4

If everything worked correctly, we should shortly see our changes on the production site, and our source code in Visual Studio Online. Let’s log back into Visual Studio Online, and check to see what has occurred after committing our changes.

build5

First, in the ‘CODE’ tab, we can see that our changes were synced to the remote repository – the “value3” change is visible in the code explorer view.

build7

Next, let’s check the ‘BUILD’ tab. Sure enough, there’s a new build there created a minute ago – it has a green tick, so the code was successfully compiled on the hosted build server.

Great, just a couple more things – let’s check the build logs, by right-clicking the build, and selecting ‘Open’.

build9

And finally, we need to open our web site, go to the values API, and confirm that the changes are successfully deployed to production.

build10

 Conclusion

This series of articles has barely scratched the surface of the capabilities in Visual Studio Online. The next step is to schedule automated tests, set up deployment slots, and customize the build process.

However, I hope this brief look at Visual Studio Online’s features is enough to encourage you to look further.

Building it in the Cloud with Visual Studio Online – Part 2

Introduction

Last week, I wrote about some of the reasons why you might want to use Visual Studio Online’s hosted build controller to build and deploy your Azure web apps. For small teams, or individuals, you can get all the benefits of a robust build server and continuous integration for your project, without the hassle of maintenance and administration. This week, I’ll build a simple MVC app, and configure it for source control and continuous integration using Visual Studio Online.

This guide is aimed at people who are somewhat familiar with Visual Studio and the basics of source control, but have not used Visual Studio Online or Team Foundation Server before.

Prerequisites

You will need Visual Studio 2013 (any version), as well as an Azure subscription to complete this walkthrough.

Creating the project

I’m going to switch things around from the usual scenario for Visual Studio Online source control, where you are forced to start with a blank project and open it in Visual Studio – if you need to know how to do that, there are plenty of excellent tutorials, like this one at: http://blogs.msdn.com/

In many scenarios you will have an existing solution that you want to add to a new Visual Studio Online repository, and this is quite simple,  but not well documented.

First, let’s create a basic MVC web app in Visual Studio, and call the app VSOWebDeploy.

NewProjectVS

We will use the Web API template, and configure it to be hosted in on Azure, like so.

SelectTemplate

Once you click OK, you will be prompted to create the Azure Web App. The app name needs to be unique, and you will need to specify a resource group, or create a new one. For the region, choose one that is near to you, and you can leave the database option as ‘No Database’. After clicking ‘OK’ the web app will be created and ready to use.

Once finished, you should be able to run the application with Ctrl-F5, using the local IIS Express server.

Configuring the web app for Visual Studio Online

Now that we have an existing application, let’s create the Visual Studio Online account that will allow us to build the solution online.

First, log into the Azure Management Portal (https://manage.windowsazure.com/), and create a new Visual Studio Online Subscription if yo don’t already have one. Go to “App Services\Visual Studio Online\Quick Create” to do this.

Select the URL (needs to be unique, e.g. <yourname>.visualstudio.com), the Azure Active Directory service and the Region closest to you, and click ‘Create Account’.

Linking the project to Visual Studio Online

For a brand new solution you can just click the ‘Open in Visual Studio’ link, and you’re ready to go. But there’s one extra step to link an existing solution to Visual Studio Online.

Basically, what I will be doing is using a local Git repository for the solution’s source control, and then pushing the master branch up to Visual Studio Online.

Go back into Visual Studio, right click the solution, and choose ‘Add to Source Control’. When prompted, choose Git as the source control provider.Choose Source Control

Next, you will see the Team Explorer – choose to see the ‘Changes’ screen, type a commit message, and click ‘Commit’ to commit the project files to local source control. At this point, your code is being tracked for changes, and the history is stored on your local drive, so let’s go one better and push those changes up to Visual Studio Online.

Click the icon in Team Explorer that looks like a power plug, and either select an empty team project, or choose to create a new team project. If doing the latter, you will be asked to supply a project name – type VSOWebDeploy  and click Create.

In a short while, the new project should be available for use. At this point, you will see a hint in the Team Explorer that you need to clone your repository to get started. This is pretty much what we need, so let’s click that link and clone our repository.

Here, it can get a bit messy, so you need to make sure you do things in the right order, or you’ll get error messages about merge ref not found, which basically means it can’t find anything to merge with.

You can resolve this in a number of ways, and if you are used to command line git, you can just force a push to the remote repository.

A simper way, though, is to go to ‘Unsynced Commits’, enter the remote repository URL, and click ‘Publish’. Your remote repository URL is in the form of https://{Your Account Name}.visualstudio.com/DefaultCollection/_git/{Your Team Project Name>} You can also find the URL by visiting your Visual Studio Online account, and going to the ‘Code’ link, if you have not yet synced to Visual Studio.

Linking the Web App to Visual Studio Online

Next, go to the web apps list, and find VSOWebDeploy in the list of Web Apps. On the dashboard, you will see a link to ‘Set up deployment from source control’.

AzureDashboard1

You will be prompted to choose location of your source code – choose Visual Studio Online, to connect the project to a Visual Studio Online account.

AzureDashboard2

After this, you will be asked to confirm the link, and that’s pretty much it – once linked, you can start using Git and Visual Studio Online builds publish your app directly from Azure, using the Hosted Build Controller.

When working on the code, make sure you remember to commit your changes regularly to the remote repository. By signing in to the Visual Studio Online account, you can explore the code, see a history of the commits, and perform various administrative tasks.

In the next post, I will finish up by setting up an automated build and continuous integration.

Azure Web Apps – Edit in Visual Studio

The ‘Edit in Visual Studio’ feature in the Azure Portal allows you to edit files on the live site, without needing to deploy or publish your app. It also contains some really useful tools for when you need to troubleshoot or configure your web app, and only have access to a web browser.

Follow these steps to enable the feature:

  1. Log into https://manage.windowsazure.com/
  2. Select the Web App, and go to the ‘Configure’ tab
  3. Scroll down to the ‘EDIT IN VISUAL STUDIO’ setting and enable it
  4. Save the changes

Now you can go the DASHBOARD link of the Web App, and find the ‘Edit in Visual Studio Online’ option in the ‘Quick Glance’ section on the right.

Apart from being able to edit files directly on the live site, you can access the following tools:

  • Git – for pushing and pulling to a local repository, or VSO
  • Output Console – for reading debug messages as the site is running
  • Console – a basic shell with some command line tools

The list of command line tools available in the console can be retrieved by typing “help”. Notable ones include the Node Package Manager (npm), Git, Powershell (ps) and msbuild, along with the usual file and directory manipulation tools.

Building it in the Cloud with Visual Studio Online

Introduction

So we recently started a new project at work, hosted on Azure. As part of the initial investigation, I stared poking around in the Azure Portal to see what new toys we might be able to play with.

One of the items under review is our source control, build and deployment tool-chain, which, frankly, is a bit outdated. Since Microsoft recently released an updated Build module for Visual Studio Online, I took the time for some further investigation.

With that in mind, let’s have a look at some of the new features, and how it can be leveraged to improve the build and deployment process.

This week, I’m looking at the Hosted Pool Build Controller

Hosted Build Controller

In the past, we have deployed and maintained our own VM’s for builds and deployment. The advantage of this is that all our source code lives on the internal network, and we are in full control of the entire build process.

Visual Studio Online can provide all that for you in a user friendly portal, where you can define your build process, set up continuous integration or a build schedule, and then receive alerts, or just review logs as needed.

The combination of Hosted Builds, Unit Testing, Web App Deployment Slots and Continuous Integration presents a powerful argument to move your builds into the cloud, but it does come with some risks and downsides.

When would you choose to use Visual Studio Online Hosted Builds?

  • You are already using VSO for source control (TFS or GIT)
  • Your application is fully hosted on Azure
  • Maintaining a separate build server is difficult or too time consuming

Downsides to Hosted  Builds:

  • You are storing your source code in the cloud, which poses a security risk
  • Limited amount of customised build steps are available
  • Support for languages, compilers and source control systems is limited
  • No administrator access to the build controller

In summary, VSO Hosted Builds are a great way to get going quickly, provided you are willing to stay ‘in the box’ with regard to compilers, build and deployment tools. There are also limitations on the size of the application (10Gb as of writing), and the list of installed applications.

The list of features is really great, and growing, so it’s well worth checking it out.

For detailed information about VSO, visit Microsoft’s Visual Studio Online page.

Part 2 of this blog post is a walkthrough on setting up a new web app and using Visual Studio Online for source control.

Part 3 describes how to implement continuous integration in the Visual Studio Online environment.