App unable to talk to Win 2012 R2 DC

  • 165 Views
  • Last Post 10 July 2017
Zoran posted this 10 May 2017

Hi guys,

I have an app which has been working with Windows 2008 R2 DCs, basically querying AD for a user attribute, but can't do it with Windows 2012 R2. The app server runs Windows 2008 R2 and the client Windows 7, and both are able to resolve SIDs S-1-18-1 and S-1-18-2. The error in the app simply says it's unable to contact Directory Services. Were there any changes in regard to Kerberos or LDAP in Windows 2012 R2?

Thanks

Zoran
Forum info: http://www.activedir.org
Problems unsubscribing? Email admin@xxxxxxxxxxxxxxxx

Order By: Standard | Newest | Votes
SmitaCarneiro posted this 10 May 2017

Resource SID compression?

https://dirteam.com/sander/2013/04/05/new-features-in-active-directory-domain-services-in-windows-server-2012-part-21-resource-sid-compression/


Smita Carneiro, GCWN
Active Directory Systems Engineer
IT Security and Policy
Ross Enterprise Center
3495 Kent Avenue, Suite 100
West Lafayette, IN 47906

show

Zoran posted this 10 May 2017

Many thanks Smita, will test it tomorrow morning and let you know if it helps.

Thanks

Zoran


Sent from my iPhone

show

Zoran posted this 18 June 2017

Just a quick update and a question on this one.
We ran wireshark on the app client, app server and DCs, and confirmed the communication happening without any issues (LDAP bind and queries from both the app client and server). In the end we figured out the problem was caused by name caching, probably DNS caching which by default expires after 24 hours, but couldn't identify where it was happening. The app server was logging errors and listing a DC by its name as unavailable, after the DC was completely removed from the DNS and DNS cache flushed from the app client and server. The app has many dependent apps and also talks to SharePoint, but the app owner was unable to shed much light on it.
The app is a .net one running on IIS. They say a Microsoft LDAP API was used to fetch attributes from AD. I noticed that the app pool service account was delegated LDAP SPNs for all the DCs, which need to be removed manually after a particular DC was demoted and deleted from AD.
My question is was it possible that Microsoft LDAP API actually cached something somewhere and is it really necessary to use LDAP delegation for a .net app running on IIS under a domain user credentials?
Many thanks for your help
Zoran


show

kool posted this 19 June 2017

Hi Zoran,

 

To answer your specific question, to the best of my knowledge the MS AD libraries do not cache DC names.

 

When a program makes an LDAP call, it has the option of specifying a domain controller name. IOW, in the path LDAP://[<dc>/]<LDAP-FQDN> the DC name is optional.

Is the app itself specifying a DC name in the LDAP path? If the app doesn’t specify a DC name, then the MS APIs use the DC locator to get a DC name. The DC locator uses DNS records to locate DCs. This is not simply the A records for the DCs themselves, there

are a number of records per-DC that get registered when a DC is promoted.

 

On each DC you will find the file “c:\Windows\system32\config\netlogon.dns” (presuming that the OS is in the default location). You should ensure that all of

these records for all of your active DCs are present in your DNS and that there aren’t old records from decommissioned DCs. Old records could be left if a DC is shut down without demoting it first. Yes, there are lots of DNS records here and any missing or

incorrect records can throw the DC locator function off.

 

Another remote possibility is that you have multiple AD sites and a site link is unreliable, but I wouldn’t worry about that until you ensure that the DNS records

are correct.

 

Cheers,

 

    Eric

 

show

Zoran posted this 21 June 2017

Hi Eric,
Thank you so much for your reply. DC names are not specified anywhere in the app as it started talking with the new and renamed DC 24 hours or so after the DC was built. I got a confirmation from a developer that the app uses System.DirectoryServices.
I cleaned up msdcs folder after the first DC was forcefully demoted, and during the testing of the new one, removed all the records of the second old DC which the app kept complaining about being unavailable since it was turned off. I also confirmed the new DC had all the records in.
In this domain, there is only 1 site, but another one is coming with several remote sites. Will see how we go about it.
I had a Microsoft field engineer with me for an entire day and we were unable to figure it out. It's a test environment so we were able to play around. He advised we build a new instance of Windows 2008 DC and force the app to talk to it in order to eliminate any issue related to Windows version.
It's all good now, but I have another 2 domains to do and was thinking it would be really great to know what had happened there.
Thanks
Zoran


show

joe posted this 21 June 2017

I wrote a whole book about effective .NET directory services programming with System.DirectoryServices a while ago. You can still get it on Amazon and it is still a useful resource even though it is more than 10 years old because this stuff hasn't really changed much since then. Your devs might find it useful.
To help understand better what is going wrong from the perspective of the app, I'd want to see how the DirectoryEntry objects are being constructed and also know what exception they are throwing when they are failing to work properly. It may also be useful to know more details of the hosting model for the application because if the DirectoryEntry is constructed with default credentials, the identity being used to connect to AD will be the identity of the current process and you'd want to ensure that it is a domain account, at least with respective to the network (for example network service or system on a domain-joined server). 
I hope that helps a bit too. If you can't repro the problem anymore, it may be difficult to figure out what was wrong but these items should help if you run into it again.
Joe K.


show

PARRIS posted this 21 June 2017

Did you check the LAN Manager Authentication levels?  https://technet.microsoft.com/en-us/library/jj852207(v=ws.11).aspx


Servers may not be able to talk by default if these are mismatched between the client and the DC. I have seen this old issue pop up a few times recently during migrations and upgrades.





Regards,



 



Mark Parris



 



Cloud | Identity | Security



 



MVP Enterprise Mobility | MCM Directory Services



Mobile: +44

7801 690596



E-mail: mark@xxxxxxxxxxxxxxxx



 

Twitter | Blog | LinkedIn | Skype

show

Zoran posted this 04 July 2017

Hi Joe and Mark and thanks for your response.
Last

week I was able to reproduce the problem in the production :0 and got

the same results as in test, but this time I replaced DC1 with the same

IP and name. Once I shut the old DC2 down, the app started complainig

about DC2 not being available. So it seems it's not a DNS related issue

as DC1 name didn't change. Once I brought DC2 up, the app started

working again. The app is some kind of a hub for a bunch of other apps

and the server cannot be rebooted, and I'm almost sure that would

resolve the issue. Anyhow, I left both DCs up and running and everthing

is okay now. In couple of days, I'll test the app against DC1 again and

will let you know how it went. We expect the app to start talking with

DC1 within 24 hours as it did in the test domain.
I'm concerned with the caching described below, but it doesn't say what happens if the DC in use goes off.
After the client locates a domain controller, the domain controller

entry is cached. If the domain controller is not in the optimal site,

the client flushes the cache after fifteen minutes, and discards the

cache entry. It then attempts to find an optimal domain controller in

the same site as the client.

https://technet.microsoft.com/en-us/library/cc961830.aspx
Below are the errors I've been getting when I turn a new DC on and turn an old one off.
Error 1:
Log Name:      Application
Source:        MyApp Internal Services Source
Date:          7/3/2017 7:43:37 PM
Event ID:      100
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      AppServer.domain.com
Description:
Timestamp: 7/3/2017 5:43:37 PM
Message: HandlingInstanceID: 123-456
An exception of type 'System.DirectoryServices.ActiveDirectory.ActiveDirectoryServerDownException' occurred and was caught.
---------------------------------------------------------------------------------------------------------------------------
07/03/2017 19:43:37
Type : System.DirectoryServices.ActiveDirectory.ActiveDirectoryServerDownException, System.DirectoryServices, Version=4.0.0.0, Culture=neutral, PublicKeyToken=blah123
Message : The server is not operational.
Name: "DC2.domain.com"
Source : System.DirectoryServices
Help link :
ErrorCode : -2147016646
Name : DC2.domain.com
Data : System.Collections.ListDictionaryInternal
TargetSite : System.Object GetPropertyValue(System.DirectoryServices.ActiveDirectory.DirectoryContext, System.DirectoryServices.DirectoryEntry, System.String)
HResult : -2146233088
Stack Trace :    at System.DirectoryServices.ActiveDirectory.PropertyManager.GetPropertyValue(DirectoryContext context, DirectoryEntry directoryEntry, String propertyName)
   at System.DirectoryServices.ActiveDirectory.DirectoryEntryManager.ExpandWellKnownDN(WellKnownDN dn)
   at System.DirectoryServices.ActiveDirectory.DomainController.getDomain()
   at System.DirectoryServices.AccountManagement.ADStoreCtx.GetGroupsMemberOf(Principal p)
   at System.DirectoryServices.AccountManagement.Principal.GetGroups()
   at ORG.My.Common.Credentials.RoleManager.get
UserRoleGroups()
   at ORG.My.MyApp.BusinessLogic.TaskStore.GetOpenTasks(Int32 pageNumber, Int32 pageSize, Boolean isGetAllOpenTasksRequired) in D:\X0\3\s\MyApp\Backend\Business\BusinessLogic\TaskStore.cs:line 109
   at ORG.My.MyApp.ServiceImplementation.ManagementServices.TaskService.GetOpenTasks(GetOpenTasksRequest request) in D:\X0\3\s\MyApp\Backend\Services\ServiceImplementation\ManagementServices\TaskService.cs:line 128
   at SyncInvokeGetOpenTasks(Object , Object[] , Object[] )
   at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
   at ORG.My.MyApp.ServiceImplementation.Infrastructure.ExceptionLoggingEnabledOperationInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs) in D:\X0\3\s\MyApp\Backend\Services\ServiceImplementation\Infrastructure\ExceptionLoggingEnabledOperationInvoker.cs:line 44
   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
Additional Info:
MachineName : AppServer
TimeStamp : 7/3/2017 5:43:37 PM
FullName : Microsoft.Practices.EnterpriseLibrary.ExceptionHandling, Version=5.0.505.0, Culture=neutral, PublicKeyToken=blah123
AppDomainName : /LM/W3SVC/5/ROOT/MyAppServices-2-131435066012405392
ThreadIdentity : DOMAIN\MyAppServiceAccount
WindowsIdentity : DOMAIN\MyAppServiceAccount   
Inner Exception
    ---------------
    Type : System.Runtime.InteropServices.COMException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=blah456
    Message : The server is not operational.
    Source : System.DirectoryServices
    Help link :
    ErrorCode : -2147016646
    Data : System.Collections.ListDictionaryInternal
    TargetSite : Void Bind(Boolean)
    HResult : -2147016646
    Stack Trace :    at System.DirectoryServices.DirectoryEntry.Bind(Boolean throwIfFail)
       at System.DirectoryServices.DirectoryEntry.Bind()
       at System.DirectoryServices.DirectoryEntry.getAdsObject()
       at System.DirectoryServices.PropertyValueCollection.PopulateList()
       at System.DirectoryServices.PropertyValueCollection..ctor(DirectoryEntry entry, String propertyName)
       at System.DirectoryServices.PropertyCollection.get
Item(String propertyName)
       at System.DirectoryServices.ActiveDirectory.PropertyManager.GetPropertyValue(DirectoryContext context, DirectoryEntry directoryEntry, String propertyName)
Category: General Category
Priority: 0
EventId: 100
Severity: Error
Title:Enterprise Library Exception Handling
Machine: AppServer
App Domain: /LM/W3SVC/5/ROOT/MyAppServices-2-123
ProcessId: 6444
Process Name: c:\windows\system32\inetsrv\w3wp.exe
Thread Name:
Win32 ThreadId:6424
Extended Properties:
Event Xml: etc
Error 2:
Log Name:      Application
Source:        MyApp Internal Services Source
Date:          7/3/2017 7:43:37 PM
Event ID:      0
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      AppServer.domain.com
Description:
The

description for Event ID 0 from source MyApp Internal Services Source

cannot be found. Either the component that raises this event is not

installed on your local computer or the installation is corrupted. You

show

Zoran posted this 05 July 2017

Just to add, the app is built on .Net Framework 4.0.30319 and it's running on IIS7 Windows 2008 R2 server. I've opened a support case with Microsoft, under ASP.NET 4, I hope that's the right address, so weill let you know if they come up with something.
Thanks
Zoran

jeremyts posted this 05 July 2017

Is this an in house app, or 3rd party? If 3rd party, have they provided any suggestions? Some applications, such as Oracle Weblogic service,

potentially requires an application restart in order to “re-home” to another DNS server and Domain Controller when doing DC maintenance. I wonder if there is something odd like this about this application. As this is an IIS application, if it has its own Application

Pool, have you tried to Recycle that after the DC change?

 

Cheers,

Jeremy

 

show

Zoran posted this 05 July 2017

Hi Jeremy and thanks for your response.
This is an in house developed app. I'll have the code they use to query AD probably later today and I'll post it here. I didn't do anything in prod, but several weeks ago while playing with it in test, we did recycle its app pool and actually even rebooted the server, but it didn't help. I've just asked the developer to check if the app uses SSL or plain LDAP to eliminate any issues with the server certificate caching as described here
https://support.microsoft.com/en-us/help/839514/a-client-computer-cannot-authenticate-to-a-domain-controller-that-is-r
although they say that the cert caching would be flushed at the server reboot, and in our case a reboot didn't help.
If no luck, I'll schedule a reboot over the following weekend and see if that helps, hoping the app would pick the PDCe. Otherwise I need to come in and shutdown the old DC2 and reboot the app server, which is what I may do if we make no progress in the next couple of days.
Thanks
Zoran


show

Zoran posted this 05 July 2017

At line 31 the exception is thrown:

var currentUserGroups = UserPrincipal.Current.GetGroups().ToList();

***

using System;
using System.Collections.Generic;
using System.DirectoryServices.AccountManagement;
using System.Linq;
using System.Security.Principal;

namespace MyNamespace
{
    public class RoleManager
    {
        /// <summary>
        ///     UserPrincipal.Current seems not thread safe
        /// </summary>
        private static readonly object _lock = new object();

        private static RoleManager _instance;

        public RoleManager()
        {
            AppDomain.CurrentDomain.SetPrincipalPolicy(PrincipalPolicy.WindowsPrincipal);
        }

        /// <summary>
        /// </summary>
        public List<string> UserRoleGroups
        {
            get
            {
                lock (_lock)
                {
                    var currentUserGroups = UserPrincipal.Current.GetGroups().ToList();
                    var list = currentUserGroups.Select(x => x.Name).ToList();
                    return list;
                }
            }
        }

        public bool IsUserInRole(string roleName)
        {          
            lock (_lock)
            {
                return new WindowsPrincipal(WindowsIdentity.GetCurrent()).IsInRole(roleName);
            }
        }

        public string UserPersonnelNumber
        {
            get
            {
                lock (_lock)
                {
                    var user = UserPrincipal.Current;

                    return user.EmployeeId;
                }
            }
        }

        public string UserDisplayName
        {
            get
            {
                lock (_lock)
                {
                    var user = UserPrincipal.Current;

                    return user.DisplayName;
                }
            }
        }

        public string UserSamAccountName
        {
            get
            {
                lock (_lock)
                {
                    var user = UserPrincipal.Current;

                    return user.SamAccountName;
                }
            }
        }

        /// <summary>
        ///     Not recommended when used with Unit Test Impersonation
        /// </summary>
        public List<string> UserRoles
        {
            get
            {
                lock (_lock)
                {
                    var currentUserGroups = UserPrincipal.Current.GetAuthorizationGroups().ToList();
                    var list = currentUserGroups.Select(x => x.Name).ToList();
                    return list;
                }
            }
        }

        public static RoleManager GetInstance()
        {
            return _instance ?? (_instance = new RoleManager());
        }
    }
}

jeremyts posted this 05 July 2017

I’m no expert, but I “think” your  guys are using buggy .NET code. From a very quick browse through, they’d get more reliable output using tokenGroups such

as an example found here:

https://blog.codeinside.eu/2017/03/02/howto-get-user-information-and-groups-from-ad/

 

Whilst it’s written in PowerShell, the code for getting the groups is a similar concept and I had heaps of problems getting this right. There’s a ton of documentation

in the script header here that I created as a reference:

http://www.jhouseconsulting.com/2013/12/20/script-to-create-a-kerberos-token-size-report-1041

 

Use a .NET decompiler like JetBrains dotPeek and have a look at the code in the Deployment Web service found here:

http://mdtcustomizations.codeplex.com/releases/view/26318. This may also assist with ideas.

 

You’d probably be best getting the guys to post to the appropriate forums here for coding help and advice along these lines:

https://social.technet.microsoft.com/forums/en-us/home

 

I don’t know how valuable this info is.

 

Cheers,

Jeremy

 

show

joe posted this 07 July 2017

Sorry this has taken me a while to respond.
First of all, I don't think there is really anything wrong with this code. The technique being used to get the user's group membership should be ok. I think it is probably easier to use the current WindowsIdentity to do this and may be less flaky but this code should work.
Looking at the source code, it looks like it is using general Windows DC locator services under the hood to discover DCs to use for these operations and is failing in doing so. This seems to suggest to me that the issue is more environmental than with the code. For example, perhaps your sites/DNS stuff is messed up after the change and AD clients are attempting to discover a DC that isn't really there?
When Windows authentication is being used in .NET, my preference for doing group membership expansion is definitely to use WindowsIdentity rather than either this code or an LDAP lookup to tokenGroups to do the same thing. It is faster, easier and generally more reliable.
Joe K.


show

Zoran posted this 08 July 2017

Hi Joe,
Thanks for the response. The tricky bit here is that the forest sits on a single site and out of approx 300 servers and 1000 PCs and users, only this server has been experiencing issues. In this domain, we have 6 apps that query AD, and the other 5 switched to new DCs as soon as I updated LDAP SPNs.
Yesterday, I used NLTest /dsgetdc:domain.com /service to see who the server talks to, and it was querying the old 2008 server for all services. Then I configured netlogon priority and weight to make the new 2012 server the preferred one. Restared dna and netlogon services on both DCs, verified the change is applied in DNS, restarted netlogon service on the app server and flushed DNS and Kerberos cache, logged off and on, and the server started switching to the new one for all services, but KDC and GC. 
This morning I confirmed that KDC service is now provided by the new server, but GC service is showing a connection to the old 2008 DC.
Then I figured out that if I demote the old 2008 DC, the app server could not refer to it as unavailable since it wouldn't be a DC anymore.
So this morning I demoted the old 2008 DC and replaced it with 2012 one, the same IP but new name. Tested the app, it's nit working, but throwing a generic error. Once I updated LDAP SPNs, the started working.
Used NLTest to check where it's connecting for AD services, and they are pointing to the new DC1, but for GC service, the app server is connecting to a DC in the child domain?! This DC also runs on 2012 but the domain was upgraded before my time.
So now I need to figure out why it can't or won't talk with the GC in its own domain.
Thanks
Zoran
Sent from my iPhone


show

Zoran posted this 10 July 2017

Jeremy, thanks for the tip, will pass it to the developer and see if he can use it.

Joe, will let the developer know about your suggestion to use WindowsIdentity and see if they can switch. I'll also ask the developers of the apps that query AD but didn't experience any issues how they do it and whether they do group membership expansion.

This morning I checked the netlogon service on the app server and it was using the 2 new DCs from its own domain. And GC service was also pointing to one of these. So I guess it seems the problem is about locating and switching to an alternate GC in a timely manner when the one in use goes off. The only difference between these apps is that they use different .Net version, but I'm not sure if that's relevant in any way. I still have a case open with Microsoft so will see if they can figure somehting out. Will post back if I discover anyting.

Thanks

Zoran

Close