Remote Development, Deployment, and Remote Debugging your first SharePoint 2007 Program

No sleep till …..WebPart!

For those who don’t get the obscure cultural reference (http://songza.com/z/ho7hzt)

I am not a developer.  I have done a little developing in the past and I have written JavaScript around SharePoint, especially with regards to the SharePoint web services.  The time has come, for me to jump, both feet, into developing against MOSS 2007 Parts.  Here are my resources and plan to get started.  This is a pretty involved process so I am going to break it down into multiple parts.

1. Setting up the Moss 2007 Development Workstation/Client
2. The Important Distinction between Remote Development and Remote Debugging
3. Configuring Remote Debugging
4. Configuring a Remote Development Environment
5. Write my/your first SharePoint program using Paul Andrew’s example
6. Configuring Remote Debugging for this first application.

Setting up a MOSS 2007 Development Environment

I used Eli Robbilard’s WSS Development Machine walkthrough below for most of this post.  There are a few additions however.

I also referenced Andrew Connell’s SharePoint Developer Tips and Tricks

So here is my detailed plan

1. Install Visual Studio 2008
2. Run the Service Pack Preparation Tool
3. Install Visual Studio 2008 SP1
4. Performed the VSeWSS on XP/Vista Registry Hack
5. Installed VSeWSS 1.2
   1. You may not want to use VSeWSS at all.. I don’t think there is a problem installing them however if you use their templates you will encounter some limitations
      1. http://scothillier.spaces.live.com/blog/cns!8F5DEA8AEA9E6FBB!671.entry
      2. Eli Robillard also recommends against it :“Not recommended. VSeWSS contains nothing to help you deploy features on anything but the machine running VS, does not support x64, and is required to open projects that it creates. So if you create a project today on a 32-bit machine, you will be unable to open it in a year when you’re building on 64-bit. This is bad.”
      3. I think I may end up using STSDEV: Simple Tools for SharePoint 2007 Development. It seems all the top Moss devs use this instead so why form bad habits.
      4. Installed the WSS 3.0 and MOSS 2007 SDKs
6. Configured the Remote Debugging Monitor on the Remote Server / Target

Install Visual Studio 2008

Ran the Service Pack Preparation Tool

http://www.microsoft.com/downloads/details.aspx?FamilyId=A494B0E0-EB07-4FF1-A21C-A4663E456D9D&displaylang=en

Install Visual Studio 2008 Service Pack 1 – While installing VS2008 SP1 I ran into an Error Code 1603 which I could not solve. I have detailed all of the fixes I tried here.

Installer – http://www.microsoft.com/downloads/details.aspx?FamilyId=FBEE1648-7106-44A7-9649-6D9F6D58056E&displaylang=en

Redistributable Packaged ISO http://www.microsoft.com/downloads/details.aspx?FamilyId=27673C47-B3B5-4C67-BD99-84E525B5CE61&displaylang=en

Installing VSeWSS 1.2 on  Windows XP/Vista using the well known registry “hack”

http://blogs.msdn.com/sharepoint/archive/2008/06/04/announcing-the-vsewss-version-1-2.aspxs

If you are using Windows XP/Vista and try to install the extension, it will display the following error:

The product can only be installed if Windows SharePoint Services 3.0 has been installed first.

Instead of replacing your OS with Windows 2003, launching a Virtual PC image preloaded with SharePoint, or trying to install SharePoint on Vista, you can simply create a registry entry to fool the extension installer as if you have installed WSS on your XP/Vista machine. Before following these simple steps, as with any registry modification, do so with caution at your own risk. You can create a registry backup if necessary.

Basically — under the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\12.0 registry key — you need to add the following string value: SharePoint="Installed".

Here is the step-by-step instruction:

1. Run regedit from command prompt or Start > Run.
2. Navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\12.0.
3. If you cannot find the “12.0″ folder, right click on the Web Server Extensions, select New > Key from the popup menu, and enter 12.0 as the new key name.
4. Right click the “12.0″ key/folder, select New > String Value from the popup menu, and enter SharePoint as the name.
5. Double click the newly created SharePoint entry, and enter Installed as the value data.
6. Now you can try to install the SharePoint extension for VS 2008.

Install Visual Studio extensions for Windows SharePoint Services (VSeWSS)

Installed the WSS 3.0 and MOSS 2007 SDKs

Additional Utilities and Tools

References:

http://www.andrewconnell.com/blog/archive/2008/10/07/So-whats-on-your-SharePoint-development-machine.aspx

http://weblogs.asp.net/erobillard/archive/2007/02/23/build-a-sharepoint-development-machine.aspx

• Windows PowerShell
• Notepad++
• DevExpress’ CodeRush & Refactor! Pro (not free)
  • Visual Studio CodeRush/Refactor Tools for SharePoint Developers
  • Visual Studio SharePoint Tool Window
• TortoiseSVN – Subversion source control client Windows Explorer shell add in
  • DeleteSVN.reg – little menu hack that removes all the Subversion files from a directory tree… great for sending a project to a client.
  • VisualSVN – adds Subversion support for (not free)
• WinMerge
• Red Gate’s Reflector
• Spence’s Application Pool Manager
• Spence’s add WSP registry hack
• SharePoint Manager 2007
• Fiddler
• TestDriven.net
• MbUnit
• Utility WSP’s
  • LogViewer.wsp – easy to read SharePoint log files
  • SharePointWarmupJob.wsp – I like mine more than the warmup scripts
  • SharePointDebugger.wsp – Feature that turns off custom ASP.NET errors
• CAML IntelliSense
• Internet Explorer Developer Toolbar
• BDC Metadata Manager
• GacUtil
• WinGac 1 or WinGac 2

The Important Distinction between Remote Development and Remote Debugging

Perhaps this is very obvious for those experienced .net and web developers but for a newbie like me it was very confusing.  Not that I didn’t understand that they were different processes, it was just that I thought that enabling one would enable the other.  My tiny developer knowledge led me to foolishly assume, “well if you can debug remotely, surely you can develop on that same platform”  with SharePoint and .net this is not the case.  I have seen these two issues confused in other blogs, especially when it comes to googling for information on them.

What I have discovered is that while all the pieces of Remote Development and Debugging are not individually difficult to configure , the “round-trip”, that being; remotely developing a SharePoint program, deploying it to a remote server or VM, and debugging it from the local workstation/client, can be frustratingly difficult to set up.

When I am done with this post I will most likely build myself a WS2003 VM and a WS2008 VM with Visual Studio installed on the VM and develop on those, because getting the “round trip” working took me about 3-4 days and is probably not worth the trouble.  However, by pushing through this task I learned a lot about SharePoint development and that was really the point.

So, here I will demonstrate how to configure each, then how to run your first SharePoint program, and then Debug it using remote debugging along with all the errors and issues I dealt with along the way.

Configuring the Client and Host for Remote Debugging

Once you have Visual Studio up and running its time to configure Remote Debugging.

Eli’s and Ross Dargan’s post combined with the official documentation http://msdn.microsoft.com/en-us/library/bt727f1t.aspx are enough to cover this topic however I cover here a few points that are not listed in either.

Eli mentions the following before deferring to Ross’s blog post

Step 10: Configure remote debugging.

Once SharePoint is installed, share the folder with SharePoint’s assemblies (*.DLL). You may also want to copy the assemblies to your development machine:
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI

Share the server’s web root and enable Read/Write.
C:\inetpub\wwwroot\

I am not sure this is necessary at this point.  I think Eli makes the assumption that you will be deploying custom applications to the 12 Hive or deploying asp.net pages to wwwroot.  I was able to attach to the w3wp.exe worker process without sharing these folders.  One of the goals of this post is to delineate clearly between the remote debugging and development processes and their  requirements.  The suggested shares will be useful however when you begin to develop custom applications as noted later in this post.

Unable to Connect to the Microsoft Visual Studio Remote Debugging Monitor

When trying to attach to the remote debugger, you may receive this error.  “Unable to connect to the Microsoft Visual Studio Remote Debugging Monitor named xxxxx “Error: The Visual Studio Remote Debugger service on the target computer cannot connect back to this computer”

The solution to this issue is to give the Development account rights to “Log On As A Service” in the Local Security Policy of the Development Workstation.  This MSDN article has the full walkthrough.  I provide some screenshots.

http://msdn.microsoft.com/en-us/library/ms164725.aspx

The following error may also occur when running the Visual Studio 2008 Debugger Configuration Wizard.  The dialog tells you how to fix the problem, so once again I only provide some screenshots.

Once we have the Remote Debugging Monitor installed on the Remote Target.  Select Tools > Options in the Visual Studio Remote Debugging Monitor and copy the Server Name (if you know the IP by heart you can use that instead)

It is now time to test that that everything is working correctly.

Open Visual Studio and select Debug > Attach To Process

Eli’s post goes over this so I am providing some reference screenshots

Finally when attempting to connect to the remote debugger you may receive a Firewall Dialog.  If you are using a VM based environment such as the one I use may have to Unblock remote debugging from any computer as the Host Machine / Development Workstation may not be on the same subnet.  In my case I had a VMWare virtual network adapter on the same subnet, but I didn’t want any problems so I still chose that option.

So now we have the processes on the remote machine available.  However, since I used multiple App Pools I have multiple w3wp.exe processes.  I you use Web Gardens you will most likely have many many more.  So which process do you choose? I went to my remote machine MossFront and opened up IIS and the Windows Task Manager to take a look at both of the processes.  By default, process monitor does not include the PID (Process Identifier Column) so to enable the column select View > Select Columns > and enable the checkbox for PID (Process Identifier) > Click OK

Now we can see the PID of both W3.  Because I am not going to deploy solutions to my Central Admin Web App, I am going to choose the w3wp.exe supporting my Portal which has a PID 2704

I am not really qualified to answer this question, however my guess is that you should choose the w3wp.exe that is running with the App Pool Identity of your build location site.  I could use some help here if someone has a more precise answer.

I received this error below, I am not sure if this is normal or not.   I only found one reference to this error http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=120421

I believe I should have specified “Managed” when selecting the process to attach to using the Select… button

If no errors are thrown after clicking Attach then we have successfully attached to the process.  We have therefore tested that we can Debug Something remotely.  The reason I emphasize this is because here is where things got really tricky for me, and where I want to attempt to save others some pain.

What we have just accomplished is setting up the mechanism to remote debug applications on a remote machine.  However the application we tried to debug was w3wp.exe, the WSS process.  The problem with using this process as an example is that all of the hard work of building, deployment, managing Code Access Security (CAC), the Global Assembly Cache (GAC) has already been done!  Most of those who will encounter Remote Debugging will use it to debug something they have written and deployed, not something Microsoft has written and deployed.  This is the example that many remote debugging walkthroughs use.  Unfortunately, using this process as the test process greatly oversimplifies the difficulties with debugging programs that you have written, built, and deployed yourself.  Therefore, what follows is a walkthrough that I have sewn together from many sources.  In this walkthrough we will set up the client to write a small SharePoint Program, Deploy It, Run it locally on the target, and then Run/Debug it from the remote machine.  I like to refer to this as the “round-trip” of SharePoint Development and have not seen an example that covers all the stages.

Configuring the Client/Workstation and Host for Remote Development

What I am coming to learn about SharePoint 2007 is that there really is no such thing as “Remote Development”.  What you have instead is the ability to develop locally against the SharePoint Libraries, and then build/deploy remotely and debug remotely.

Corey Roth’s blog post is the piece of information that finally made this clear to me.

If you are developing on a desktop, you will need to do is copy the SharePoint assemblies to your computer.  We can easily get these from a deployed SharePoint server.

C:\Program Files\Common Files\microsoft shared\Web Server Extensions\12

Go to the above path and then go into the ISAPI folder.  Copy all of the DLLs from this folder onto your desktop machine in the corresponding folder name.  If you don’t have that folder, create it.  You may also copy them to the Global Assembly Cache as well.

Corey says;

You may also copy them to the Global Assembly Cache as well.

This is a bit confusing to me because it makes it sound like an option.  I have not found documentation stating if this is required.  For the purposes of this post I am going to assume it is.

First lets copy over the .DLLs.  I located the directory on my VM using the Shared folder I created when configuring Remote Debugging.  Copied all of them and then went back to the host to paste them, and found something interesting.

What I had discovered is that something I had installed had created a “12 hive” on my machine even though I didn’t have WSS of MOSS 2007 installed.  I believe it was most likely one of the SDKs (WSS MOSS), but can’t confirm yet.  The interesting thing was that it was a only a shell of the 12 hive.  When I looked inside the ISAPI folder I only found XML files not DLLs.

Once again I was confused but decided to press on and copy all the DLLs into the local ISAPI folder.  I even copied the _VTI_ADM and _VTI_AUT folder because they contained ADMIN.DLL and AUTHOR.DLL.

So now my local shell “12 hive” looked like this

I used WinGac to install all of these DLLs to the GAC.

My output was:

Number of assemblies processed = 19
Number of assemblies installed = 14
Number of failures = 5

Failed to add assembly C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI\OWSSVR.DLL to the cache:   The module was expected to contain an assembly manifest.
Failed to add assembly C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI\SHTML.DLL to the cache:   The module was expected to contain an assembly manifest.
Failed to add assembly C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI\SSOCLI.DLL to the cache:   The module was expected to contain an assembly manifest.
Failed to add assembly C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI\_VTI_ADM\ADMIN.DLL to the cache:   The module was expected to contain an assembly manifest.
Failed to add assembly C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI\_VTI_AUT\AUTHOR.DLL to the cache:   The module was expected to contain an assembly manifest.

I don’t know enough about the requirements to know if this will cause me problems later or what I am missing out on for not being able to add these 5 assemblies to the GAC, but pressing on…

First SharePoint Program

Paul Andrew has a good post about beginning SharePoint Development.  The first program he lists is an enhanced version of one from the WSS SDK.  I like both versions.  The WSS one has the benefit of not assuming any development knowledge beyond starting up Visual Studio 2008.  However, Paul’s Code is more advanced and from what I gathered from the comments, superior.  There is, also a functional difference.  the SDK example merely counts the number of lists in a site collection, while Paul’s version enumerates each web (SPWeb) in the site collection(SPSite) and enumerates each list in each site. Therefore, my example will combine the context from the SDK, Paul’s Code as well as providing some additional code that Paul assumed would be present.  To be clear I am not questioning Paul’s assumptions.  It is reasonable to assume some Visual Studio / .Net experience.

http://blogs.msdn.com/pandrew/archive/2008/05/01/getting-started-with-sharepoint-development.aspx

http://msdn.microsoft.com/en-us/library/ms438026.aspx

Here are the two versions in C#

SDK Version:

static void Main(string[] args)
{
    SPSite siteCollection = new SPSite("http://Server_Name");
    SPWebCollection sites = siteCollection.AllWebs;

    foreach (SPWeb site in sites)
    {
        SPListCollection lists =  site.Lists;

        Console.WriteLine("Site: " + site.Name + "  Lists: "
           + lists.Count.ToString());
    }

    Console.WriteLine("Press ENTER to continue");
    Console.ReadLine();
}

Paul Version:

static void Main(string[] args)

{

// Update to your server name

using (SPSite siteCollection = new SPSite(http://localhost))

{

SPWebCollection site = siteCollection.AllWebs;

foreach (SPWeb web in site)

{

try

{

SPListCollection lists = web.Lists;

Console.WriteLine(“Site: {0} Lists: {1}”,

web.Name, lists.Count.ToString());

foreach (SPList list in lists)

{

Console.WriteLine(“List: {0} {1}”,

list.Title, list.ID.ToString());

}

}

//catch (Exception)

//{

// // handle

// throw;

//}

finally

{

web.Dispose();

}

}

} // dispose is called on site as a result of using()

Console.WriteLine(“Press ENTER to continue”);

Console.ReadLine();

}

The WSS SDK provides the following walkthrough which is useful for us novices:

How to: Create a Console Application

This programming task describes how to create a console application in Microsoft Visual Studio 2005. The example displays the number of lists within a site collection.

Users must be administrators on the computer where the script is executed in order to run a console application in the context of Windows SharePoint Services.

To create a console application in Visual Studio 2005

1. On the File menu in Visual Studio 2005, point to New and then click Project.
2. In the New Project dialog box, select a language, and then select Windows in the Project Types box.
3. In the Templates box, select Console Application.
4. In the Location box, type the path to where to create the application, and then click OK.
5. In Solution Explorer, right-click the References node, and then click Add Reference on the shortcut menu.
6. On the .NET tab of the Add Reference dialog box, select Windows SharePoint Services in the list of components, and then click OK.

   JUSTIN’S NOTE: One thing that is important to note is that these assemblies will not be available in the .NET tab of the Add Reference dialog if you are developing remotely,\ regardless of the location of the assemblies (remote or local).  In addition, if you deploy your own assemblies they will not appear in the .NET tab even if you add them to the GAC.  Below is a list of discussions I found on this topic.  The short answer is that the contents of the .NET tab are not built through interrogation of the GAC as you might expect, but are path-based.
   

   http://weblogs.asp.net/jdanforth/archive/2003/12/16/43841.aspx

   http://support.microsoft.com/default.aspx/kb/306149

   http://www.mail-archive.com/advanced-dotnet@discuss.develop.com/msg09216.html

7. Because, the assemblies are not in the .NET tab as directed by the WSS SDK walkthrough we will select them from the location on the local Hard Drive using the Browse Tab

   

   

8. In the .vb or .cs file, add a using directive for the Microsoft.SharePoint namespace, as follows.

   using Microsoft.SharePoint;

Now we’ll depart from the WSS SDK version of the code and use Paul’s code

Here is Paul’s code combined with the code that Visual Studio 2008 added for me when using the Console Application Template.  I have also changed the Server_Name to target the Server created in my Small vFarm Walkthrough.

1 using System;

2 using System.Collections.Generic;

3 using System.Text;

4 using Microsoft.SharePoint;

5

6 namespace ConsoleApplication2

7 {

8 class Program

9 {

10 static void Main(string[] args)

11 {

12 // Update to your server name

13 using (SPSite siteCollection = new SPSite(“http://MossFront”))

14 {

15 SPWebCollection site = siteCollection.AllWebs;

16 foreach (SPWeb web in site)

17 {

18 try

19 {

20 SPListCollection lists = web.Lists;

21 Console.WriteLine(“Site: {0} Lists: {1}”,

22 web.Name, lists.Count.ToString());

23

24 foreach (SPList list in lists)

25 {

26 Console.WriteLine(“List: {0} {1}”,

27 list.Title, list.ID.ToString());

28 }

29 }

30 //catch (Exception)

31 //{

32 //    // handle

33 //    throw;

34 //}

35

36 finally

37 {

38 web.Dispose();

39 }

40 }

41 } // dispose is called on site as a result of using()

42 Console.WriteLine(“Press ENTER to continue”);

43 Console.ReadLine();

44 }

45 }

46 }

After adding the code, edit the properties of your project and move the Build Location to a location on the remote server.  For my first example I have created a shared folder called “Dev Builds” on my remote VM.

Save your build configuration.

Once our code is correct both the WSS SDK and Paul’s post recommend the following.  The reason is that both assume local development.  This is not a failure of these resources, just something I had to learn the hard way.  But just for the heck of it try running it as suggested.

9.  Click Start on the Debug menu or press F5 to run the code.

This will most likely fail with a security exception.  Perhaps something that looks like this:

System.Security.SecurityException was unhandled

Message: Request for the permission of type ‘Microsoft.SharePoint.Security.SharePointPermission, Microsoft.SharePoint.Security, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c’ failed.

We will get around this later in the walkthrough.  For now lets build our first program on the remote server and run it so that we can feel good about having accomplished something.

Build your solution by selecting Build > Build Solution or pressing Ctrl + Shift + B

Because we configured a remote build location the program should show up in the location we specified.  My shared location was C:\Dev Builds and there you can see the ConsoleApplication2.exe has appeared.

I believe you need to be an Administrator for this program to run correctly so perform a RunAs and use an account with local Administrator privileges.

I find it odd that both Paul Andrew’s and the WSS SDK’s example of this test program leave something quite important out. Where is the the expected output.  Clearly, they have put a lot of effort into evangelizing SharePoint development, but I have never read a programming book or article that did not post the expected output.  Because of this when trying to extrapolate their example from local to remote development I really couldn’t tell if it was or wasn’t working at first.  Without further delay, here was my output (yours will vary as this first program enumerates lists)

The Output:

SUCCESS!!!!…for now

After developing this simple program remotely deploying it and getting it to run I was pretty excited, but I wanted to figure out how to debug it remotely and get F5 in visual studio to work as I wasn’t looking forward to creating a build every time I wanted to test.  So we continue with….

Remote Debugging a Simple SharePoint 2007 Program

So when we ran the program locally it worked, which was a huge success.  But now, what about remote debugging it?

Oddly when trying to remote debug the same program we just successfully ran locally, using the same permissions, I received the following error.  This seemed really odd to me as I assumed the program would run with the identity of the Remote Debugging Monitor.  Running the program locally using that account worked, while doing the same from the remote machine fails.  What gives?  My error is copied below:

Request for the permission of type ‘Microsoft.SharePoint.Security.SharePointPermission, Microsoft.SharePoint.Security, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c’ failed.

It looks like I am not the only person to run into this error.  Chris Green appeared to run into this problem and provides a solution

Chris’ solution worked perfectly for me.  I don’t know if this is secure or not, my guess is not, but I have to some more research on what the most secure way to debug remotely is.  I have two caveats to Chris’ list of steps

1.  The .NET Framework 2.0 Configuration that Chris references is installed with the SDKs so make sure they are installed on your development sever/VM

2.  If you are new to developing with strong names.  check out these posts that got me through.  You have to create the strong name, along with a key file.  When the solution is built the keyfile is built with it.  Below are the references I used to guide me through this process.  In addition I have included some discussion of why using your keyfile in this way is a bad idea.

How to: Sign an Assembly with a Strong Name

MOSS 2007 and Code Access Security

Custom Code Access Security for Web Parts

Jan Tielens – Common Security Pitfalls for Web Part Developers

SharePoint remote debugging macro

Remote Debugging SharePoint Webparts in Moss

Remote Debugging a Web Application

Remote Debugging a SharePoint Server 2007 Workflow

I hope that this post presents a simple view of the the SharePoint 2007 Remote Development and Debugging “round trip”.  I haven’t seen an example that ties all of the pieces together in one example, and I hope this is a start.  I blogged this as I did it for the first time so I am sure that there are inaccuracies.  Please correct me if you find something in this post that you disagree with or have a better approach.

---

Possibly related posts: (automatically generated)

• SharePoint – Development, Deployment and Detritus
• Starting as a Sharepoint Developer
• Deciphering that Burning Fire