Tag: Governance

Quota Management and Storage Reports in SharePoint 2010

A few years ago I wrote an article about how to enable and work with the Quota Management features in SharePoint 2007 (click here for article) which proved to be a popular post.  Quota Management is a pretty important topic when it comes to SharePoint Governance and overall maintenance of the platform.  While the overall Quota Management features in SharePoint 2010 were maintained, there was one big feature left out when SharePoint first shipped, and that was the “Storage space allocation” page also known as StoreMon.aspx page that was available to Site Collection administrators from the Site Settings page.  

New Storage Metrics

With the release of SharePoint 2010 SP1 (download here) the feature returns, but in a much different format and vastly improved.  The page was renamed “Storage Metrics” and it is a gold mine of information since it provides a way for Administrators to navigate through the content locations on the site and provides details for the item’s Total Size, % of Parent, % of Site Quota, and Last Modified Date.  This makes it easy for administrators to identify where content is concentrated, and can also show an exceptionally large lists, libraries, folders, and documents. 

There was one aspect of this that I thought was helpful in 2007 that is no longer supported, and that is the ability to view the number of versions of a given document right from the report.  In many cases I’ve seen versioning turned without any limits, and some popular documents might have 1,000s of versions.  The report used to provide a way to find those exceptions so that they could be cleaned up.

 

Performance Improvements

From what I understand, it was removed because it proved to be extremely resource intensive and information was gathered in real-time so it could cause service stability issues in very large environments.  With its return is a completely revamped gathering process that relies on timer jobs, titled Storage Metrics Processing, resulting in much faster page loads and no risk of crashing the server just by viewing the report.  These jobs will pull data every 5 minutes but like all timer jobs, the frequency can be adjusted to better meet your needs and environment.  For larger environments, it might be a good idea to reduce that frequency to avoid the extra overhead.

Configuring Quotas

As with the 2007 version, this feature is only available if quotas are enabled.  In cases where quotas are not currently being used and proper limits managed, the safest bet is to establish a quota that cannot be met.  This will enable the features without the risk of triggering a warning or locking a site that exceeds the thresholds.  Locking the site is the only risk with quotas, there is no risk of data loss.

Summary

Both Farm and Site Collection Administrators should review the functionality and add its review into their content review and cleanup processes.

SharePoint Governance and Continuous Improvement

There are a number of reasons why I have seen SharePoint Governance stall in many environments, but one of the most common reasons and the one I would like to address today, is the problem of trying to take or address too much scope at once.  There are a lot of topics that roll into SharePoint Governance; the SharePoint Governance Framework we use at my company identifies over 50 distinct topics.  In most cases it is not practical to take the full list of topics and work through a governance plan in one pass.  Instead, I advocate for an iterative approach starting with a selection of core topics and finding a pace that matches your organizations cadence to continue solidifying the process.

Continuous Improvement

Continuous Improvement (CI) is a concept that relates to Lean, Kaizen or TQM practices and is intended to use small, incremental changes to bring big results instead of trying change everything at once.  There are many advantages to this approach but the one that resonate most with me is that the incremental investments could be very small, but you start to see gains quickly.  I believe this relates very well to the process and concepts of SharePoint governance.  As you start to gain momentum you are much more likely to get stakeholder involvement which is critical to any governance effort.

Selecting Topics

The selection of topics is important.  There are some topics that lay the foundation for everything else.  I typically start with the core topics that are needed to define the service and system to be implemented (or that has been implemented).  I then move to the most critical items that are needed for managing the system such as touching on the core Information Architecture, Site Topologies, Site Creation Process (who/how/where) and Permission Management.  Over time, and definitely in more mature organizations, you can start to tackle some of the more subtle topics.

Depth for Topics

When reviewing a topical area, it is important to iterate there as well.  For example, when starting an implementation stakeholders may have a decent idea what the service definitions and SLAs might look like, but this will certainly change after it is in use for a while and perhaps became mission critical for core business processes.  As these requirements and needs evolve, additional iterations are required.  This fits very well with the CI concepts mentioned above.  These iterations continue to deliver more value to the stakeholders in small increments.

Finding a Pace

Finding the pace for the governance work can be difficult.  I think the pace for change in any organization is heavily tied to the organization’s maturity.  Anyone that has been IS/IT or IS/IT management for while should be familiar with the Capability Maturity Model.  In Level 1 organizations things are completely chaotic, with ad-hoc efforts which means you are constantly chasing a moving target.  As you progress up the maturity model though, there should be more discipline and an ability to stay focused.  In these organizations it is possible to iterate more frequently and potentially to tackle more topics in parallel.  There is no doubt that more mature organizations work more efficiently.

Other Considerations

Governance should be an ongoing process and the document should be a living document that continues to see change over time as the organization and it’s needs evolve.

It is also important to note that groups should not take a shortcut and try to minimize the stakeholder involvement to simplify the process or to quicken the pace.  Stakeholder involvement, specifically non-IT, is critical to these efforts and it could be argued that you really do not have any governance without this involvement.

Summary

Governance is a key aspect to success with SharePoint and by taking the right approach, and one that is in sync with your organization’s capabilities will greatly increase the likelihood that your governance effort will be successful.

Integrating Open Source Software and Components

Over the past few years there has been tremendous progress in making many components and solutions available in the open source markets like CodePlex, SourceForge, or even individual blogs and sites.  Many of these are community driven projects supported by individuals, with some by commercial companies.  For some reason many companies avoid or strictly limit allowing these solutions to be installed or used.  When approached properly I think these solutions should be considered.  This article will cover the advantages as well as outline an approach to make take when evaluating which solutions to install and how to test them.

Advantages

The first advantage of implementing any solution is that it should make it much quicker to solve a business problem.  In some cases it might be a complete solution, like commercial software components, but at the very least it should act like an “accelerator” offering you a springboard to the final solution.  One advantage that it has over commercially packaged systems is that you have the source code in case changes, additions, or fixes need to be made. 

Another advantage is that it can expose the development team to different coding techniques so that they can better address similar problems in the future.  Microsoft has been providing sample databases and applications for years for this very purpose.  People tend to learn more by seeing examples in action versus class diagrams.

Review Project Status and Activities

Not all solutions are of the same quality and grade.  In some cases there are a few quick samples while in others there was a more formal project with full QA that has gone through multiple release cycles.  It is therefore important to review things like the product status and release number.  It is also important to review the Issues log to see how much activity there is and how quickly issues have been addressed.  You can typically get a good idea about how widely it is used.  In some cases you may be familiar with the developers which offers the advantage of discussing questions and issues directly with them.

Approach to Testing

To increase your chance of success and mitigate risk, it is important to always fully test the components to meet both functional and performance testing.  With commercial software the expectations are normally pretty high, but I would approach this as if a member of the project team produced it even if no changes were made.  This includes specialized testing like the “DisposeCheck” tests typically done to custom code that interacts with SharePoint’s API. 

Final Notes

When expectations are properly set, and the solution is fully reviewed and tested, including these open source solutions can contribute to the team’s ability to effectively deliver solutions quickly.

Related Articles

MySite Provisioning Methods

A number of times over the past few years I have stumbled into discussions (in person or online) about how to automate the creation of MySites for all users in the organization.  Creating the sites programmatically is actually pretty simple, but the real question is “Why do you want to do that?”  There are advantages and disadvantages to automating the process, but for me it almost always comes down to two big things; Governance and Business Reason. 

MySite Governance

MySites present an interesting challenge with regards to governance.  While most of the topics are outside the scope of this article there are a few important topics that relate to the number of MySites within an organization.

Storage Considerations – Even with quotas in place it is easy to see exponential growth for the storage requirements.  In larger environments with 1,000s of users serious planning needs to take place to build out a SQL environment to support the site collections.  Planning should also be done to manage the number of sites per content database to ensure long term maintainability. 

When you provision all of the sites at once all of the planning has to be done up front, where conversely if you provision the sites slowly over time you spend a little time planning out the long term assumptions and then tweak the strategy over time as the sites and their usage evolves.  It is much easier to make corrections with the slow approach.

User Support and Training – A MySite is very different than an email account which is something nearly all computer users are familiar with at this point.  The average SharePoint user has never received any formal training and has little understanding of the capabilities of a site collection.  Without proper training it is unlikely that user will be able to take advantage of any of the real benefits of the MySite leaving them to just use it as a replacement for a personal network share (see Storage Considerations above). 

In my experience site owners or administrators for traditional collaboration or department sites are much more likely to have success and less likely to need extra support.  That narrower group of people is a much better starting point, and they are also sophisticated enough to initiate the automatic provisioning process themselves.

Business Reason

Each organization should develop a user story for what the purpose of a MySite is within their organization.  Like any site collection, it can be used for many different purposes such as; Landing Page, Dashboard, Personal Site, etc.  The user story may help establish how the MySite will be used, who is expected to use it, and ultimately if customization is needed to provide the functionality and content.  The answers to those questions should help guide the decision about how to provision the sites. 

Closing

While I tend to like the go slow and make adjustments path, there are valid reasons for needing to auto-provision sites for large groups of users.  Hopefully the guidance here will help to guide the team through proper planning so that the implementation can be successful.

Related Posts

Site Topology Planning and Taxonomies

In the previous article SharePoint Site Topology Planning I discussed some of the technical implications of organizing the the sites within one or more applications, site collections, and sub-sites.  The article started to get pretty long so I decided to save the taxonomy part of the discussion for a separate article.

Organizing Sites

In the previous article I addressed different types of content and using that to help segment the sites across applications and site collections.  Within a given application it is possible to provide some meaningful segmentation by configuring managed paths. 

For example, you may segment collaboration sites with the following url structure:

  • http://collaborate.company.com – Collaboration Application
    • /Communities – Communities of Practice Managed Path
      • /Proposals – Proposal Community of Practice Site Collection
      • /Procurement -  Procurement Community of Practice Site Collection
    • /Projects – Projects Managed Path
      • /Alpha – Alpha Project Site Collection
      • /Omega – Omega Project Site Collection

Organizational Hierarchies

Most people understand hierarchies, and most businesses (at least in the west) have been organized in hierarchies for many years.  It is natural for people to think of their organization in this manner, but this may not be the best way to plan for the topology of your sites.

Traditional Intranets tend to go from the largest organizational unit down to the smallest.  There may be multiple divisions, with multiple business units, with multiple departments, with multiple teams, with people that actually do the work.  Sites or portals that go 5 or more levels deep can become very difficult to manage and even harder to use.  Modern businesses need to remain agile with teams always being redefined, combined and split up.  In  most situations it is a good idea to fight the hierarchy tendencies and strive for a flatter structure. 

From a SharePoint perspective a flatter structure with more site collections will make it easier to reorganize sites and structure versus a single site collection with 5+ sub-site layers deep.  As previously discussed Site Collections can be backed up and restored with a high level of fidelity (completeness) compared to a sub-site’s export and import options.  The key to usability and manageability is to find the right amount of segmentation and site collection structure.

Finding Sites and Content in a Flat World

An alternative to a rigid hierarchy is adopting flexible taxonomies with tagging.  Tagging provides a flexible and dynamic method of describing the content and sites that can evolve over time.  A great example of this is a site like StackOverflow compared with the rigid structure of the MSDN/TechNet Forums. The flat structure decreases the chance of duplication and provides new opportunities to view the data in new and unique ways. 

The SharePoint 2010 system full supports tagging without the need for custom or third party add-on components.  I fully expect that these will be a popular feature within the new version. 

Summary

Following the guidance between the two articles you should be able to properly plan your site topology.  Assumptions and business decisions do change, but if you establish the right level of granularity with applications and site collections you will be able to migrate and relocate things as needed.

Related Posts

SharePoint Site Topology Planning

This is the next in a series of articles addressing core SharePoint implementation topics.  I hope that this is valuable to both groups looking to implement SharePoint for the first time as well as groups planning planning for an upgrade or migration to SharePoint 2010.

A Series of Containers

I tend to think about SharePoint from a container perspective.  Each of the containers has a set of settings and features that can be configured and administered for all of the containers within.  It is important to understand the boundaries of each level so that you can create a site topology that meets all of your objectives.  I’ll cover a sub-set of these I typically consider while planning the site topology.  The containers I’m going to address include:

Farm – In the context of this article, all of the Applications, Site Collections, and Sites hosted by one or more connected SharePoint servers.  I say “connected” because it is possible, and fairly likely that most organizations will have more than one Farm (Dev, Staging, Production for things like Intranet, Extranet, Internet, etc).  A farm has one or more content Applications plus additional applications for things like Central Administration.

Application – An application would be the top level address which maps to an application in IIS.  For example http://sharepoint.  An application has one or more Site Collections.  The application level is also the level where you have the opportunity to identify one or more content databases for storing the content on the SQL server.

Site Collection – A site collection has one or more Sites also referred to as Webs or Sub-Sites.

Sub-Sites or Webs – This is the smallest container which resides under and can be managed by the Site Collection.

Types of Content

The first step is identify what types of content or site(s) you expect to host.

  • Internet
  • Extranet
  • Intranet
  • MySites
  • Company or Divisional Portal(s)
  • Web Content Management (Publishing)
  • Electronic Content Management (Document Management)
  • Business Intelligence
  • Project Management
  • Team Collaboration
  • Application Hosting

Not all of those types of content apply to every organization, but it should be pretty clear that the content, the update frequency and the manner in which it is used and managed can vary quite a bit from type to type. 

Considerations of Multiple Applications

Very small companies or workgroups may be able to get by with all of the types of content hosted in a single Application, but for anything larger there should be plans to segment it to multiple Applications in the farm.

Here are some considerations when using multiple Applications:

Authentication Model – What type of authorization model will be used?  Options include Anonymous, Windows Integration (NTLM or Kerberos), and Forms Based Authentication (FBA).  Since an Application can only support a single authorization mode, that can dictate additional applications.

Edit:    Anders Rask was kind enough to point out some changes to the authorization model in 2010 that make the previous statements incorrect.

In 2007:  Options include Anonymous, Windows Integration (NTLM or Kerberos), and Forms Based Authentication (FBA).  Since an Application can only support a single authorization mode, that can dictate additional applications.

In 2010: There are two high level options; Classic which supports Windows Authentication only and Claims Based which supports one or more different providers.  Additional information can be found on TechNet’s article Plan Authentication Methods.

Sessions – When a user visits an application a session is created.  It is important to understand that their session is for a specific IIS Application so if they visit the main company portal and then click to see the MySite they may be asked to authenticate again.  With Anonymous this should not be an issue.  With Windows Integrated this should not be an issue if 1) users are using Internet Explorer 2) they access the site(s) with the same account they use to access their computer and 3) their browser is properly configured to pass logged on user information. 

Application Scoped Solutions – Solutions can be scoped to specific Applications which can provide some flexibility in deploying new features.  In a large environment it is important to only show features to the areas where they apply.

Considerations for Site Collections

The site collection has some important boundaries to consider when deciding to use one versus multiple site collections.  They include:

Amount of Data – It is important to keep your site collections at a maintainable level.  There is no hard limit, but as Site Collections grow over 40GB they can get more difficult to maintain and will take longer to restore.  SQL Server tuning becomes more critical with larger databases.  It is a good idea to segment your content across multiple site collections (and across multiple content databases) in a manner that makes sense.

SharePoint Groups – SharePoint Groups can be a good way to organize users, especially when they do not map to functional areas or groups otherwise managed in Active Directory.  These groups are defined and contained within a given Site Collection which means if you want to use them in multiple site collections they have to be duplicated and maintained separately which can be problematic.

Site Collection Administrator – A site collection administrator has great power and control within the site collection container.  A Site Collection administrator can choose themes, manage all security within the site collection, manage activated solutions and potentially deploy other customizations if policy permits.  Enabling content owners should be a top priority, and the Site Collection provides a good container for that.

Quota Management – Quotas are set at the Site Collection level so if granular quota management is required for billing or charge back purposes it may have some impact on how site collections are segmented. 

Navigation – The default SharePoint navigation provider does not span site collections (and therefore applications) which can make a standardized or unified navigation scheme difficult to maintain.  This tends to work fine for team collaboration sites, but can be cumbersome when you need to link many site collections.

Content Types – I think Content Types were one of the most important changes introduced with WSS 3.0 and MOSS.  An entire overview of content types is outside the scope of this article, but keep in mind that within the current release Content Types are created and maintained within a Site Collection.  If a Content Type applies to multiple site collections then it needs to be duplicated.  SharePoint 2010 will support farm level content types which will remove the need to duplicate them.

Profiles (WSS Only) – If you are using WSS it is important to understand that the profiles are stored at the Site Collection level.  If you add custom attributes they will need to be added to all site collections they apply to.

Implications on Backup and Recovery

There are a few different methods to backup and recover SharePoint content.  Within the context of this article it is important to understand the difference between the commands that stsadm provides. 

Site Collection Backup and Restore – Considering the boundaries previously discussed, there is a lot of extra content stored in the top level site of a site collection.  Doing an stsadm backup will provide a high fidelity snapshot of the content, configuration, workflows, and other customizations.  It is important to note that if you are moving the site collection between applications or farms that you will need to install any solutions or dependencies referenced in the current location.

Sub-Site or Web Export and Import – The Export process offered for Sub-Sites is great for archiving, but it does not provide the fidelity needed to move sites around.  It will not save workflow, features, solutions or alerts.  I’ve also had inconsistent results with DataViews on sites being migrated in this manner.

If you think you will need to migrate the content or want flexibility, it would be in your best interest to consider using more Site Collections rather than deeply nested Sub-Sites.

Upgrade and Migration Considerations

The purpose and content within a site collection or site can evolve over time.  Some sites that started very narrow in purpose may change and now warrant their own Site Collection or Application.  When preparing for a migration or upgrade it is a good time to run through this exercise again to validate the assumptions and decisions that were previously made or overlooked.  While it may make the move more difficult the changes will pay dividends over the coming years offering a system better tuned to the user’s needs and more maintainable by the site owners and farm administrators.

Related Posts

SharePoint Dev and Customization Paths

Microsoft’s technology stack has traditionally offered a lot of good options leaving the developer or architects to choose the best path.  Even still there there have been people camping out on one side or the other.  In the early days when VB first came out the hardcore devs didn’t take it seriously, and I remember hearing a lot of negative comments from traditional C++ developers.  VB devs didn’t have to understand concepts like threading and memory allocation and the VB code was bloated leading to “poor performance.”  While those statements are true, not every deliverable has the same grade requirements and in many cases businesses were more than happy with the code that was delivered faster and for less money than what C++ developers could provide.  I think this is a very good analogy to the current SharePoint development and customization paths.  Microsoft has worked hard to position SharePoint as a tool that can be leveraged and customized by power users as well as developers.  I think this is both a strength and a weakness; a strength because there are options, but a weakness because many of the people doing the work do not understand the implications of their decisions.

Understand the Scope of the Solution

The first thing I think about when I get a request is the scope of the solution.  I think through the questions below and choose a direction that makes sense in order to meet the requirements in a cost effective manner. 

Who is the audience? – The larger the scope of the audience, the more robust the solution should be.  A solution created for a work group is probably very different than one used enterprise wide.

How many users will need to use it? – Similar to the previous question, but a little more concrete.  Do you expect 10 concurrent users, 100 or 1000?  Company or enterprise wide means different things in different environments.

Where will the solution be used? – Deploying the solution on a single Project or Team Site is very different than deploying it to 100 sites, which is very different than deploying to the front page of a company’s Intranet.  If it is for a single use the Light Dev and Customization path can make sense.  If it is meant to be reusable, then it warrants more time and attention to make sure that it is the best it can be.  If it is use on a top level or heavily used page or site then performance is even more critical meaning a more robust solution should be developed.

How long is it expected to be in use? – Try to have some understanding of how long the solution is intended to live.  Assumptions may change, but make an informed decision based on the information you have at the time.  If a newly acquired third party tool will be providing that functionality in the next 10-12 months there is no need to overbuild the solution.  Likewise, don’t take short cuts on a critical project that can undermine peoples’ understanding of the capabilities of the platform.

Who will maintain the solution? – This is a tricky one.  Not every company has a large staff of qualified developers waiting in the wings.  There is normally a large backlog of work waiting for them.  If you are a consultant then you might look at whether you are handing over the support to the internal team or if your group will continue to maintain it.  The group isn’t capable of maintaining a solution and doesn’t have the tools and environment in place to do the work then full scale development may be a bad idea.

Are there performance requirements? – If there are performance requirements then those should be planned for from the start.  Stats like page load times, and an agreement on what constitutes the end of a page load are important.  In most cases I have not seen this requirement.

Light Development and Customization

Light development and customization includes features and solutions dealing with SharePoint Designer, DataViews, Content Query Web Part and the Content Editor Web. 

More than anything I see this being for the benefit of Power Users and site owners who can apply customizations within the scope of a site collection.  While there are many that want to outlaw SharePoint Designer and these solutions in their environment, I think they risk eroding the value and impact of their SharePoint platform.  I see SharePoint as a tool that enables content/site owners without needing to rely on a central IT group.

I tend focus on this part of the stack when dealing with solutions targeted to small teams or with small requests that do not need a full blown application.  There are definitely lots of opportunities for simple, high impact, one off solutions. 

This however is probably not the appropriate set of tools for org wide workflows, complicated forms, or even displays on heavily trafficked pages.  The Content Query Web Part (CQWP) for example is not the most efficient way to aggregate content so pulling something like project status information from 50+ project sub-sites is likely to end in frustration.

Robust Development

Robust development includes features and solutions dealing with Visual Studio built applications packaged and deployed via the solution framework in SharePoint. 

This really should be the starting point when reviewing any project of real significance.  The developer has a lot more flexibility and control in what can be implemented and it is much easier to centrally track and manage what is deployed to the server. 

I think that it is important to note that poorly developed VS solutions are at least as dangerous as anything that can be done using the Light Development and Customization technologies and techniques.  Failing to dispose of objects or inappropriate CAML queries can bring down a server or cause all kinds of havoc.

Mixing Approaches

There will be times when it is necessary to blend the two approaches.  Within a given project there are features that are core to the solution and some that are in more of a supporting role.  Admin or reporting features are a good example of that, where using a DataView may be easier to maintain than a custom Web Part to display content.

The Business Does Not Always Understand

Customers, project stakeholders, and other decision makers do not always know what is best.  I’ve seen many cases where even sophisticated development shops do not fully understand SharePoint development paths and fully leave the decision up to the developer.  Like any project using any technology, risks should be identified and discussed before things get too far.  It is irresponsible to deliver a solution that you know cannot be supported or maintained after you sneak out the back door at the end of the engagement. 

Administrator’s Perspective

From an administration perspective, I do not want to manage more solutions than I need to.  There is a balance between wanting to know what is deployed to the environment and wanting to touch everything that is installed.  If there is a fairly ridged change control process in place it can be time consuming to submit and deploy minor updates to small, narrowly scoped solutions.  Again, my approach comes back to the scope of the solution preferring to focus on widely scoped, high impact solutions.

Related Posts

%d bloggers like this: