Microsoft is aggressively working on making SharePoint better and with release of SharePoint 2016 RTM, many organizations are planning to migrate to cloud or upgrade their On-Premises installation. Recently I have been working on lots of migration related projects (some of them from SharePoint 2003 to 2013 - yes, there are customers out there who are still using SPS 2003 :) One of the client had a single site collection and content database for their Portal and team sites. The content database was about to reach 200 GB and they wanted to split the major sub sites and site collections into individual site collections with its own database. Client hired a third party vendor and they did the job for them. We are not sure what approach was followed to achieve this. However, after the split we saw the following issue.

Issue:

Most or all of the following should be true to make sure that you are facing exactly the same issue.

• sub sites were exported / imported or moved using some third party tool as site collections into their individual database.
• Root site collection (before the split) which hosted the sub sites had publishing feature and infrastructure enabled (site collection scope)
• sub sites (before the split) had publishing feature enabled (web scope)

After the split, when you create a sub site under the site collection and try to active Publishing feature (web scope), you get the following error:

Server Error in '/' Application

Runtime Error

Description: An application error occurred on the server. The current custom settings for this application prevent the details of the application error from being viewed remotely (for security reasons). It could, however, be viewed by browsers running on the local server machine.

Details: To enable the details of this specific error message to be viewable on remote machines, please create a <customErrors> tag within a "web.config" configuration file located in the root directory of the current web application. This <customErrors> tag should then have its "mode" attribute set to "Off".

When we check the ULS logs, we get the following errors:

Add users as SharePoint Shell Admin for specific site collection or content database

Recently we received a concern from one of our team members that they are not able to run PowerShell scripts against a specific web application or site collection. They were trying to Backup the site collection and using Backup-SPSite cmdlet.

However they were not able to do so and got the following error:

Backup-SPSite : Cannot open database "<database_name>" requested by the login.

The login failed.

Login failed for the user '<domain\username>'.

At line:1 char:14

We ran the following command to add the user as PS Shell Admin

Add-SPShellAdmin -UserName domain\username

However, even after doing so they were getting login failed error when trying to run PowerShell commands. The problem was occurring because as the error stated, the user's login did not have SharePoint_Shell_Access rights on the database. You can easily check this by checking the database security properties (from SQL management studio) and check the permissions of the account. If the account is not listed under Database -> Security -> Users, then it is obvious that the account does not have any permission on that content database.

When we run "Add-SPShellAdmin -UserName domain\username", it only added the user account under SharePoint Config database with SharePoint_Shell_Access rights. User's account was not added on any of the content databases with the Shell rights. That’s why when the user tried to backup a site collection, they were getting login failed error. For more details on what the Add-SPShellAdmin cmdlet, please refer to this article - https://technet.microsoft.com/en-us/library/ff607596.aspx

So what we had to do was to specifically grant SharePoint_Shell_Access rights by passing on the content database name like in the example below:

 Get-SPDatabase | ?{$_.Name -eq "WSS_Content"} | Add-SPShellAdmin -Username DOMAIN\Username

This solved the problem. Also, in case the user again complains for login failed error, you need to take a look at the site where he is trying to run the command and make sure that the account does have SharePoint_Shell_Access permissions on that database that is holding that site collection. If not, run Add-SPShellAdmin cmdlet by specifying the database. As always, its best practice to have the least permissions and review the SharePoint Shell Access rights using the command - Get-SPShellAdmin and remove any user accounts that are not supposed to be there.

A specified logon session does not exist. It may already have been terminated.

Issue:
On a Windows 2008 R2 or Windows 7 machine, you open Task Scheduler. You create a new task and the following conditions are used in the task.

• The account used to run the task is another service account (i.e. author or the user who is trying to save the task) is different from the user who will be running the task
• Option of “Run whether user is logged on or not” is selected.
• Option of “Do not store password. The task will only have access to local computer resources.” is not selected.

When you try to save the task with the above general settings, we receive error.

Error Message:
An error has occurred for task <TaskName>. Error message: The following error was reported: A specified logon session does not exist. It may already have been terminated.

Cause:
This occurs because the local security policy has the following setting:
Network access: Do not allow storage of passwords and credentials for network authentication - Enabled.

Resolution:
To verify whether the security policy is causing the issue. Log on to the machine where you are facing the problem.

• Start -> Administrative Tools -> Local Security Policy
• Security Settings -> Local Policies -> Security Options
• Network access: Do not allow storage of passwords and credentials for network authentication setting should be Enabled.

Just FYI, the corresponding registry key for this setting can be found here:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
Key Name: disabledomaincreds
Current Key Value: 1
The value of “1” means that the policy is enabled. It must be “0”.

NOTE: If you change the registry value to “0”, you should be able to save the task. Do not make registry changes unless you are absolutely sure. Only use it for checking it this resolves the issue or not on a test machine. Actual resolution is to update the group policy and disable the security policy mentioned above. It will take care of this registry.
Disable the policy from domain controller and run a group policy update.

SharePoint 2010 - Crawling stuck, crawl component in "Recovering" state

From past few days we started this issue out of now where. The crawler is stuck. When we browse to the Search Administration page, we see that the Crawler component is stuck in "Recovering" state and due to that Query component also reported a status as "Not Responding". Propogation status was shown as "Query server not responding"

We checked the SharePoint logs and found errors as below

file copy failed (error 0x80070005: Access is denied.   0x80070005, source c:\program files\microsoft office servers\14.0\data\office server\applications\GUID-crawl-0\projects\portal_content\indexer\cifiles\0001001A.ci, destination \\machinename\GUID-query-0\GUID-query-0\Projects\Portal_Content\Indexer\CiFiles\0000.0001001A.ci.cp)  [indexpropagator.cxx:403]  d:\office\source\search\native\ytrip\tripoli\propagation\indexpropagator.cxx

Exception thrown (0x80070005: [needs message] info 0)                           [indexpropagator.cxx:409]  d:\office\source\search\native\ytrip\tripoli\propagation\indexpropagator.cxx

In Event Viewer, we found Event ID: 2587 with the description that "The following conditions are currently affecting index propogation to this server for search service application "Service Application Name": 1. Query 0 has been disabled so that crawls can continue. It may be recovered via the Restart-SPEnterpriseSearchQueryComponent command in PowerShell

The code - 0x80070005 states that the access was denied and the account accessing the folder did not have permissions.

I granted permissions for the service and crawler account on c:\program files\microsoft office servers\14.0\data\office server\applications\
But this did not resolve the issue. This meant that apart from the location mentioned in the logs, the permission issue was somewhere else too. In such cases, process monitor does help. I took a process monitor and found that there were errors related to permissions on C:\ProgramData and few of its SharePoint subfolders. After granting the service and crawler account permission (Full Control) over the C:\ProgramData folder, the status of the components was not showing as "Online".

In case you face such errors, try giving permissions for the search service and crawler account (in case they are different) over the C:\ProgramData folder and check. If not, take help from Process Monitor to find out where the permission issue is.

Do not remove, add or restart any component unless you are absolutely sure that permissions or any of the below mentioned causes is not an issue.

Few other things to verify:

• Ensure that the search service is started. Open service manager (Start -> Run -> services.msc) and ensure that "SharePoint Server Search 14" is started.
• Re-enter the password for the service account for the service which you checked in the above step.
• Ensure that the search service instance is online by checking the same by using a powershell/stsadm command
• stsadm -o osearch -action list  This will show if the search instance is online or not on the servers.
• http://blogs.technet.com/b/patrick_heyde/archive/2009/12/07/sharepoint-2010-get-search-configuration-over-powershell.aspx

Update: Recently we noticed that restarting the server running the component would get the component back to "Online" state. Before restarting any services or running PowerShell cmdlet, try rebooting the server.

SharePoint 2010: User Profile Service not starting, remains in Stopped state

Issue:Not able to start User Profile Synchronization Service from “Services on Servers” page in Central Administration.

Symptoms:You have created a User Profile Service Application and now would like the User profile synchronization service to start only on specific servers. So, you go to Central Administration –> Application Management –> Services on Server.
On this page, you try to start the “User Profile Synchronization Service”. It stalls on “Starting” state for a minute or two and then reverts back to “Stopped” state.

Error Messages(s):When the status of the service returns to “Stopped” state following errors are logged in event viewer.

Log Name:      Application
Source:        Microsoft Resource Management Service
Date:          xx/xx/xxxx x:xx:xx xx
Event ID:      0
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer: xxxxxxxx
Description:
Service cannot be started. System.Data.SqlClient.SqlException: Could not find stored procedure 'RegisterService'.
   at Microsoft.ResourceManagement.WindowsHostService.OnStart(String[] args)
   at System.ServiceProcess.ServiceBase.ServiceQueuedMainCallback(Object state)


Log Name:      Application
Source:        Forefront Identity Manager
Date:          xx/xx/xxxx x:xx:xx xx
Event ID:      3
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      xxxxxxxx
Description:
.Net SqlClient Data Provider: System.Data.SqlClient.SqlException: Could not find stored procedure 'RegisterService'.
   at Microsoft.ResourceManagement.Utilities.ExceptionManager.ThrowException(Exception exception)
   at Microsoft.ResourceManagement.Data.Exception.DataAccessExceptionManager.ThrowException(SqlException innerException)
   at Microsoft.ResourceManagement.Data.DataAccess.RegisterService(String hostName)
   at Microsoft.ResourceManagement.Workflow.Hosting.HostActivator.RegisterService(String hostName)
   at Microsoft.ResourceManagement.Workflow.Hosting.HostActivator.Initialize()
   at Microsoft.ResourceManagement.WebServices.ResourceManagementServiceHostFactory.CreateServiceHost(String constructorString, Uri[] baseAddresses)
   at Microsoft.ResourceManagement.WindowsHostService.OnStart(String[] args)


Log Name:      Application
Source:        Microsoft.ResourceManagement.ServiceHealthSource
Date:          xx/xx/xxxx x:xx:xx xx
Event ID:      2
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      xxxxxxxx
Description:
The Forefront Identity Manager Service could not bind to its endpoints.  This failure prevents clients from communicating with the Web services.

A most likely cause for the failure is another service, possibly another instance of Forefront Identity Manager Service, has already bound to the endpoint.  Another, less likely cause, is that the account under which the service runs does not have permission to bind to endpoints.

Ensure that no other processes have bound to that endpoint and that the service account has permission to bind endpoints.  Further, check the application configuration file to ensure the Forefront Identity Manager Service is binding to the correct endpoints.


Log Name:      Application
Source:        Microsoft.ResourceManagement.ServiceHealthSource
Date:          xx/xx/xxxx x:xx:xx xx
Event ID:      26
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      xxxxxxxx
Description:
The Forefront Identity Manager Service was not able to initialize a timer necessary for supporting the execution of workflows.

Upon startup, the Forefront Identity Manager Service must initialize and set a timer to support workflow execution.  If this timer fails to get created, workflows will not run successfully and there is no recovery other than to stop and start the Forefront Identity Manager Service. Restart the Forefront Identity Manager Service.

Cause:Incorrect permissions and configuration of the account that is used to run the Profile Synchronization Service.

Resolution:

1. Add the account to local administrator’s group on the server where you are trying to start the service
2. Restart the “SharePoint 2010 Timer Service” on the server.
3. Ensure that you do not start the “Forefront Identity Manager” service manually. Although, if you want, you can change the service start up type to “Automatic (Delayed)”.
4. Now, try to start the service from Central Administration –> Application Management –> Services on Server.