Hello Y’all,
This is an interesting problem that I worked a couple of weeks ago, and it was a thought-provoking mystery until we figured out what was happening, as always, with a little help from my friends.
The mystery and the problem:
Here is the deal, no new computer objects that were being discovered by SCCM were showing up under
Assets and ComplianceOverviewDevices, but why?
We started looking for clues under the Data Discovery Manager logs, DDM.log:
*** Exec spGetNextIDInARange N’NextIds’, N’NextMachineID’, N’System_DISC’, 16777216, 25165823, N’ItemKey’ SMS_DISCOVERY_DATA_MANAGER
*** [42000][229][Microsoft][SQL Server Native Client 11.0][SQL Server]The SELECT permission was denied on the object ‘NextIds’, database ‘CM_PRI’, schema ‘dbo’. SMS_DISCOVERY_DATA_MANAGER
WARNING – GetNextIDInARange() failed to execute SMS_DISCOVERY_DATA_MANAGER CDiscoveryDataManager::ProcessDDR – could not get next available item key. SMS_DISCOVERY_DATA_MANAGER
CDiscoverDataManager::ProcessDDRs_PS – SQL problem detected. Will retry later. SMS_DISCOVERY_DATA_MANAGER CDiscoverDataManager::THREAD_ProcessNonUserDDRs – Failed to manage files in inbox. Will retry in at least 60 seconds SMS_DISCOVERY_DATA_MANAGER
Refreshing site settings….. SMS_DISCOVERY_DATA_MANAGER
Processing file adum3ayb.DDR SMS_DISCOVERY_DATA_MANAGER *** Exec spGetNextIDInARange N’NextIds_G’, N’NextUserID’, N’User_DISC’, 2063597568, 2080374783, N’ItemKey’ SMS_DISCOVERY_DATA_MANAGER
*** [42000][229][Microsoft][SQL Server Native Client 11.0][SQL Server]The SELECT permission was denied on the object ‘NextIds_G’, database ‘CM_PRI, schema ‘dbo’. SMS_DISCOVERY_DATA_MANAGER
WARNING – GetNextIDInARange() failed to execute SMS_DISCOVERY_DATA_MANAGER CDiscoveryDataManager::ProcessDDR – could not get next available item key. SMS_DISCOVERY_DATA_MANAGER
CDiscoverDataManager::ProcessUserDDRs_PS – SQL problem detected. Will retry later. SMS_DISCOVERY_DATA_MANAGER CDiscoverDataManager::THREAD_ProcessUserDDRs – Failed to manage files in inbox. Will retry in at least 60 seconds SMS_DISCOVERY_DATA_MANAGER
What we were seeing was an access denied for the ‘NextIds‘ and ‘NextIds_G’. OK, we get that, without a NextId, no new computers will be added… but who was trying to select those objects? Important: the SCCM environment was a Primary Site and a Remote SQL Database, so maybe the Computer account did not have the necessary access to the database, or something corrupted the access of the Primary server computer account in the SQL database. That was the theory, and theories are nothing if you can’t test them, right?
So, let’s test the theory by removing the computer account from the SQL database, and re-adding it. Well, THAT was something I haven’t done before, and it was a great learning experience.
(SPOILER ALERT! The steps below to remove and re-add the account did not solve the problem, but they were part of the process that I went through to try solve the problem, and of course, this is the whole idea of these posts, so that even though some troubleshooting I’m showing you here did not solve the problem, it may help you to resolve yours, because although the problems and error messages may be similar, what I have learned is that understanding why you took some troubleshooting steps can be as important as solving the problem itself.
To remove and re-add the account:
- You need to be logged in via an account with SA (aka sysadmin) permissions for the database.
- PLAN! Find a maintenance window, because we’ll have to drop the connections between the Primary server and the database, and that will be downtime! Because that was the only functionality that we could see that was not working, (package and application deployment seemed ok, and software updates also ok) we didn’t want to cause any new problems while fixing the current one.
- From the Primacy Site Server we manually stopped all SMS services, and as a good reminder, try to use the command preinst /stopsite, because this will trigger a site reset once you start the server again. (You don’t have to believe me, you can read this article that explains it.)
- From the SQL box, we restarted the SQL Server service, and monitored the connection using the sp_who2, very useful Stored procedure built-in that will show us all the current connections. Once you don’t see the primary server connections, we are ready to go.
Here is an example from my lab on how to remove, and re-add the computer account:
Once all the Services are stopped, we will delete the computer account from the User’s Security Node for the SCCM database: in my example, the CMCBTX-PRI-01.
![]()
Now we will remove the account from the Security Node
![]()
Next we will re-add the account, right click under logins and under login name, we use domaincomputername$ i.e CMCBTX-PRI-01$
![]()
Once added, right click and under properties > Server Roles, make sure to add the account as SYSADMIN
![]()
Under User Mapping, select the SCCM Database and make sure to use the default schema as dbo, and as a role, add it with db_owner.
![]()
After that, you should be able to start the SCCM services at the Primary site server.
I warned you, right? Yes, it did not work…
And we don’t give up just because one action plan didn’t work. Time to bring the big guns, the SQL Profiler. That dude will tell us what is happening, right? Partially… Damn spoil alerts again!
SQL profiler is a snitch, but this time, it was hiding something from us. This is where the mystery gets very interesting, at least for me.
So we ran a SQL Profiler, no big deal, selected the SQL server name, connect, event selection, show all events, show all columns, right click errors and warnings, select all, right click Security Audit and Sessions, deselect all and right click TSQL and Store Procedures and select all. Easy, right? More than we need, but sometimes in SQL Profiler, it’s good to have that extra.
So, we ran SQL Profiler for about 5 mins, just to be sure, and stopped the trace. I know, I know, I should have used SQL Profiler from the beginning, and I did, and I am showing you what I saw…
User Error Message The SELECT permission was denied on the object 'NextIds_G', database 'CM_PRI', schema 'dbo'. SMS_DISCOVERY_DATA_MANAGER S-1-9-3-1264295285-1190618126-1402209955-3677970905 S-1-9-3-1264295285-1190618126-1402209955-3677970905
User Error Message The INSERT permission was denied on the object 'NextIds_G', database 'CM_PRI', schema 'dbo'. SMS_DISCOVERY_DATA_MANAGER S-1-9-3-1264295285-1190618126-1402209955-3677970905 S-1-9-3-1264295285-1190618126-1402209955-3677970905
Interesting, right? So tell me: who is S-1-9-3-1264295285-1190618126-1402209955-3677970905?
That is the big question. He’s the one to blame for those access denied errors and the root cause of me not being able to sleep at night, but, what, who, how, where, when?
I was expecting to see an account, like the computer account and/or user account. No, it was not that easy to find that GUID, and it was not in the Active Directory, we checked, so what on earth is happening?
Researching the Interwebs we found this very interesting function that would convert GUIDs to SIDs. Now we’re on the right track, so let’s keep going. Here is the function, in case you don’t already have it in your database:
CREATE FUNCTION fn_SIDToString
(
@BinSID AS VARBINARY(100)
)
RETURNS VARCHAR(100)
AS BEGIN
IF LEN(@BinSID) % 4 <> 0 RETURN(NULL)
DECLARE @StringSID VARCHAR(100)
DECLARE @i AS INT
DECLARE @j AS INT
SELECT @StringSID = 'S-'
+ CONVERT(VARCHAR, CONVERT(INT, CONVERT(VARBINARY, SUBSTRING(@BinSID, 1, 1))))
SELECT @StringSID = @StringSID + '-'
+ CONVERT(VARCHAR, CONVERT(INT, CONVERT(VARBINARY, SUBSTRING(@BinSID, 3, 6))))
SET @j = 9
SET @i = LEN(@BinSID)
WHILE @j < @i
BEGIN
DECLARE @val BINARY(4)
SELECT @val = SUBSTRING(@BinSID, @j, 4)
SELECT @StringSID = @StringSID + '-'
+ CONVERT(VARCHAR, CONVERT(BIGINT, CONVERT(VARBINARY, REVERSE(CONVERT(VARBINARY, @val)))))
SET @j = @j + 4
END
RETURN ( @StringSID )
END
After creating the function. we tried to look inside the server principals, but nothing was found there with that SID.
Any more ideas? Yes! Always! What about EXEC sp_helprolemember?
And what if we could use the Function we inserted before to convert the MemberSID into a GUI?
Yes! SQL Again!
--Step 1 - Creating a TEMP TABLE to later on, insert the value of the sp_helprolemember SPROC
CREATE TABLE #temp_table(DBROLE varchar(30), MemberName varchar(30), MemberSID varbinary(200))
-- Step 2 - Inseting the output of the sp_helprolemember into the TEMP TABLE
INSERT INTO #temp_table
EXEC sp_helprolemember
-- Step 3 - Using the function we previously created to convert the Hexadecimal into the readable SID
select *, dbo.fn_SIDToString(MemberSID) from #temp_table
-- Last Step after you are done with step 3, Dropping the Table.
drop table #temp_table
That part of the mystery solved, we were now able to see what account was using that GUID, but why was this account ‘smsdbuser_ReadWrite’ being used, and why was this account not showing up as ‘smsdbuser_ReadWrite’? Instead, it was showing as S-1-9-3-1264295285-1190618126-1402209955-3677970905. We needed answers, so with assistance of an SQL guy, the great PFE George Manson, we solved the problem by opening one of the store procedures that were being used in the process of the data discovery. The answer was in the sproc!
spGetNextIDInARange_internal ALTER PROCEDURE [dbo].[spGetNextIDInARange_internal]
@NextIDsTableName sysname,
@IDName nvarchar(30),
@ArchTableName sysname,
@RangeStart int,
@RangeEnd int,
@ColumnName sysname = N'ItemKey'
-- This uses dynamic SQL so default permission chaining doesn't work.
WITH EXECUTE AS 'smsdbuser_ReadWrite' -- needed to provide permissions for dynamic SQL statements
And we can see that the stored procedure is calling the account with the code: WITH EXECUTE AS ‘smsdbuser_ReadWrite’
If you want to read more about the EXECUTE AS argument here is the documentation.
So that answered a lot of the questions I had, because another thing I was seeing in the SQL profiler traces was that the computer account was being used, to the point of the error. When the error started happening, the account changed to that GUID. we just needed to add the accesses back, but which ones? Looking at my lab, that was not altered: db_datareader and db_datawriter. Under the security node for the CM_PRI database > Security > Users >, right click on the smsdbuser_ReadWrite and in Membership, add those accesses.
Told ya, the answer was boring, but not the journey! Root cause? Someone looked at that guy and said, you should not have access to anything, I never heard of you, I will fix you, bye bye accesses, and…from what we could see…bye bye discovery.
Thanks to Umair Khan and Arjun Mohan for their time and help to tackle this problem and the wife for correcting the text.
See you next time!
Renato Pacheco | Support Engineer | Microsoft Configuration Manager
Disclaimer: This posting is provided “AS IS” with no warranties and confers no rights.
