Windows - a brief guide inside

[Paul Adams]

Windows NT 101

Foreword
"NT" is the generic name for the family of Windows which stemmed from the original "Windows NT" operating system.
Windows 2000, XP & 2003 are NT variants, when Microsoft switched from a number-based versioning system for its product line.
Windows 95, 98 & ME are the other family of Windows which are in reality shells on top of MS-DOS (often referred to as "Win9x").

The brief articles that follow deal exclusively with the NT family, while there may be some similarities with the Win9x functionality, there are a huge number of differences.
They are meant merely as an introduction to certain topics for those who might be interested, and as a kind of revision aid for me.

The main sources of information are Windows Internals Fourth Edition (Russinovich & Solomon) and MSDN online.

This is a whistle-stop tour of some Windows fundamentals, there's not a huge amount of real detail, just enough to whet your appetite if you feel so inclined to research further.

Summary
The areas I intend to touch upon are:
1. The role of the BIOS
2. The boot process from a hard disk
3. The logon process - processes that run automatically
4. Processes, threads and memory
5. Services
6. Drivers
7. Debugging
8. Windows Vista

[If you think there are errors, please PM me and I'll correct as necessary.]

[Paul Adams]

=====================================================================
1. The role of the BIOS
=====================================================================
The first thing that has to occur when you turn a computer on is the well-known "Power On Self Test" (POST), this function is provided by the BIOS (Basic Input/Output System).
The BIOS does not rely on a CPU, RAM or a hard disk - it remembers its settings through non-volatile EEPROM (Electronically Erasable Programmable Read Only Memory).

The BIOS is responsible for providing the details of the system hardware at the lowest level - it can interrogate all its known interfaces, find devices attached to them and "enumerate" them.
Some of these settings are remembered between restarts, others are "plug and play" and are detected automatically every time you turn the computer on.

In older machines, the BIOS was a very basic beast - everything had to be specified manually including providing information on the cylinder, head and sector count of your hard disk, and if you got this information wrong you could write garbage over your data.
The hard disks and BIOSes we deal with today are a lot more flexible and self-governing to save us a lot of this kind of hassle.

As well as configuring the BIOS manually, we had to concern ourselves with which slot we installed hardware and setting the "jumpers" correctly on the devices so they did not conflict with each other.
We also had to remember the settings we assigned to the hardware so that we could tell the drivers for the operating system (such as interrupt, address and DMA channel).
The introduction of "plug and play" made life a lot easier - plug a device in and turn the computer back on and the BIOS (or suitable operating system) would be able to "soft" set the values as required.

Nowawdays the BIOS can have options for overclocking, RAID, alerts based on fan speed or temperatures, even built-in audio CD playing functionality.
A huge number of motherboards are also coming with onboard sound and graphics chipsets, also controlled through the BIOS.

So the BIOS enumerates the hardware, maybe tests the memory, but how does it know where the operating system (or systems) are in order to provide you with your interface?
And how does it know whether it should try to boot from the floppy disk or CD you happen to have inserted?

The BIOS is given a list of boot preferences - in what order it should check the different channels/devices to see who has "bootable" media installed.
The "boot" process can occcur from floppy disks, CDs, DVDs, hard disks, USB devices and even the network.

Starting at the top of the list, each device is checked for bootability (the actual process differs depending on the type of device) and once one is found the BIOS loads the necessary code and begins to execute it, ending its direct involvement in the startup process.

[Paul Adams]

=====================================================================
2. The "boot" process from a hard disk
=====================================================================
As the most common boot process takes place from a hard disk, this is what I shall focus on.

A hard disk has a Master Boot Record (MBR) stored on its first sector, the master boot code examines the master partition table for an extended partition, and if it finds one it begins to recursively examine the extended partition table to find all logical partitions (they are chained, so the chain must be followed to its end).

Once all of the extended partition information is resolved, the master boot code attempts to load the partition boot sector of the primary partition that is marked as "active" (there can be only one marked active at any time) - if there are no such partitions (and no further bootable devices are found) then an error is presented indicating that the system is not bootable.

The active primary partition will have a pointer to NTLDR (the NT Loader) which is the first tangible thing a user can see if they were to browse the disk.

Up until now the only disk services available have been provided by the "INT 13" BIOS interface, just enough to be able to locate, load and execute the boot sector, which has the ability to understand the Windows file systems on the partitions on the system just enough to load NTLDR.
Also, the system has been running in "real mode", which means no virtual-to-phyiscal memory addressing can take place, and only the first 1MiB of memory is addressable.

NTLDR switches to "protected mode" (allows all physical memory to be addressed), enables paging (virtualisation of memory addresses) and opens C:\BOOT.INI (this file cannot reside elsewhere) to find a list of operating system options.

While Windows Vista supports the NTLDR boot loader, its default is a new type of boot loader which does not rely on BOOT.INI (in fact this file is ignored).
The boot loader type can be changed by using the tool "fixntfs".

Protected mode with paging enabled is the mode in which Windows executes normal operation.

If there is just one operating system and one entry in BOOT.INI, no menu is presented by default.
In the event of a dual-boot (or even multi-boot) setup there will be more than one entry in BOOT.INI, each entry points to a disk/partition/folder where the operating system resides, with one is marked as default.

An example BOOT.INI entry:
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP Professional x64 Edition" /fastdetect /NoExecute=OptOut

In the event that a legacy (Win9x) Windows operating system is selected from NTDLR, BOOTSECT.DOS is loaded which contains the original boot sector for Windows 95/98/ME.

There are many documented switches available for boot options in BOOT.INI - the most common are probably /3GB and /DEBUG (I will touch on these later)

There can only be 1 "non NT" operating system in the boot loader menu, and there is a limit of 10 options in total.

If a version of Windows is already installed on a system and a Windows installation is started again then this second version can be installed to any partition with room - BOOT.INI will get amended automatically to add this new boot option to the list.

Another key file on the system partition is NTDETECT.COM which collects system information during the boot, and some systems using SCSI devices may have NTBOOTDD.SYS which contains a SCSI driver.

Notes on 64-bit Windows
Some versions of Windows are 64-bit and as such require a more recent version of NTLDR (to allow 64-bit protected mode), so if you install Windows XP x64 Edition first and follow it with an installation of Windows XP (32-bit) then the first installation will not be bootable as the NTLDR will be regressed to the version which only understands 32-bit versions of the kernel.

The 64-bit NTLDR & NTDETECT are backwardsly-compatible so are able to boot either a 32-bit or 64-bit version of Windows and it is safe to install a 64-bit version after a 32-bit version.

In the event where a system becomes unbootable, it could be:
- BOOT.INI needs fixing to remove invalid entries (maybe a second Windows installation was removed by just formatting its partition)
- the MBR needs repairing (see the above note, or a legacy or non-Windows OS overwrote it)
- essential boot files are missing or corrupt (disk corruption, virus or other OS possibly to blame)

To recover a system that has lost its boot options but the Windows folders appear to be present, the best option is to boot from the Windows installation CD/DVD - from there you can choose a repair installation, or go into the Recovery Console and run one or more of the repair tools FIXBOOT, FIXMBR or BOOTCFG.

Based on the outcome of processing BOOT.INI, NTLDR then performs the following actions:
- loads HAL.DLL (the kernel-mode DLL which interfaces NTOSKRNL and drivers with the hardware: HAL = "Hardware Abstraction Layer")
- loads BOOTVID.DLL
- loads boot-start device drivers
- loads NTOSKRNL.EXE (the kernel - initializes the executive subsystems and boot and system-start drivers, and runs SMSS.EXE)

NTOSKRNL.EXE also contains the Windows loading splash screen which some people like to hack to change but can be a risky business.

SMSS.EXE is the Session Manager SubSystem, it loads the Windows subsystem including WIN32K.SYS and CSRSS.EXE and starts WINLOGON.EXE
- WIN32K.SYS is the kernel mode part of the Windows subsystem
- CSRSS is the "Client Server Runtime Subsystem Service", the user-mode part of the Windows subsystem
- WINLOGON starts the Service Control Manager (SCM), Local Security Authority Subsystem Service (LSASS) and presents the authentication prompt

SMSS, CSRSS and WINLOGON can be seen as running processes in Task Manager - if the "session 0" instance of any of these processes is terminated, Windows will reboot as they are absolutely vital.

WINLOGON is what intercepts the "Secure Attention Sequence" (SAS) - you know this as "CTRL ALT DELETE".
If there is no user logged on currently then the logon prompt is presented.
If there is a user logged and the Welcome Screen is used then the Task Manager appears
If there is a user logged and the "classic logon" method is used then "Windows Security" appears, with options such as changing the password and locking the workstation.

It is a security feature to ensure that no other process can ever intercept that key combination, and you are really authenticating to Windows and not a keylogging program written by someone else.

If a system is configured in the BIOS to boot from a CD/DVD drive before a hard disk, and you happen to have a Windows NT 4.0 CD in the drive, then Windows Setup will launch from there rather than your Windows installation booting as normal.
This has caused me many "what the hell?!?" moments after reconfiguring the OS by installing something from the CD that requires a restart.

But how can this be prevented?
If the BIOS checks the CD and sees it is bootable, it will boot off it.

With Windows 2000 onwards the CD/DVD still gets booted, however the bootstrap now checks the hard disks for bootable primary partitions and if it finds one it presents the message "Press any key to boot from CD...".
If the user does not press a key within a few seconds, then the boot loader loads the boot code from the hard disk, rather than continuing to boot from the CD.

Of course, if there is no bootable primary partition then the CD will control the entire boot sequence.

If the computer is a member of a domain, then the client now checks for group policies with computer configuration settings applicable to it and any scripts under the following location are executed (even if a user does not log on):
Computer Configuration / Windows Settings / Scripts / Startup

[Paul Adams]

=====================================================================
3. The logon process - processes that run automatically
=====================================================================
Whether you hit CTRL-ALT-DEL or click on your icon and optionally enter your password, the user logon process is the same.

Windows calls the GINA (Graphical Identification and Authentication) to obtain the username and password, WINLOGON also generates a unique local logon SID (Security ID) for this user session which is passes to LSASS (Local Security Authentority Subsystem Service).
LSASS does the actual authentication and builds the user "logon process token" - if the authentication is successful then the SID from WINLOGON is put into the token.
Once the user's permission to log onto the machine has been verified, the group memberships that the user has are also logged in this token.

The token is used so that every time access is requested to an object, the user's group membership and permissions list does not have to be re-enumerated, wasting lots of time.

If the user account is local then the authentication is done against the local SAM (Security Account Manager) database.

If the user account is in a domain then the authentication is done securely against a domain controller (if no domain controller is present but the user has logged on successfully recently then cached credentials are used to authenticate the user).

WINLOGON then checks the following value and executes the program(s) found there:
HKLM / SOFTWARE / Microsoft / Windows NT / CurrentVersion / Winlogon / Userinit
The default is "userinit.exe".

Userinit.exe checks this registry key to see which shell (desktop) to launch:
HKLM / SOFTWARE / Microsoft / Windows NT / CurrentVersion / Winlogon / Shell
This key does not exist by default, and it defaults "explorer.exe".

Once the shell is launched, userinit exits - which is why explorer.exe appears to have no parent process in a session.

Next, the scripts from the following 3 locations are executed concurrently:

i. HKLM / Software / Microsoft / Windows / CurrentVersion / RunOnceEx
The keys exist under this location and have string values under them which are executed – the names of the values themselves are irrelevant, but the keys are processed synchronously in alphabetical order.
The values are removed from the registry immediately on being executed.

ii. User Properties / Profile / Logon Script
In the Profile tab of the properties of the user object in AD, the script referenced in the “Logon script” field is executed.
The default location is %systemroot%\sysvol\sysvol\{domain name}\scripts for any script not explicitly given a path.

iii. Group policy/policies – user configuration
The group policies applicable to the user logging on are evaluated and any scripts under User Configuration / Windows Settings / Scripts / Logon are executed.

It is possible that these 3 steps can start before any computer configuration scripts through group policy, if the user logs on as soon as the CTRL-ALT-DEL window appears – this can be prevented by enabling the following local/group policy option:
Computer Configuration\Administrative Templates\System\Logon: Always wait for the network at computer startup and logon

Processes that are to run just once and then be deleted (e.g. installation completion routines) are governed by the registry key:
HKLM / Software / Microsoft / Windows / CurrentVersion / RunOnce
The values under this location are executed synchronously in the order in which they were created.
The values are also removed from the registry immediately on being executed, unless the name begins with a “!” in which case it is removed once the process has completed.


Processes that are common for all users who log onto the local machine are governed by the registry key:
HKLM / Software / Microsoft / Windows / CurrentVersion / Run
The values under this location are executed synchronously in the order in which they were created.
Entries that begin with a “*” are executed even when the machine starts in Safe Mode.


Processes that are specific to the user currently logging on are governed by the registry key:
HKCU / Software / Microsoft / Windows / CurrentVersion / Run
The values under this location are executed synchronously in the order in which they were created.
Entries that begin with a “*” are executed even when the machine starts in Safe Mode.

Quick clarification & hint: "synchronously in the order in which [the values] were created" means they are executed one after another in "natural order" and not how they are displayed through Registry Editor (alphabetically).
The natural order can be determined by exporting the key to a text file and viewing it in Notepad.

Finally we get to the Startup folder - the shortcuts in the “All Users” and current user profiles are executed concurrently (Start Menu\Programs\Startup).
“All Users” profile is always on the local machine, but the user profile location can be on a network resource (defined in the user properties) or the local machine.


A quick note on "sessions".

Session 0 is both the session used to identify processes running as "the system" and also the one used by the user logged on at the console (or remotely with the "/console" switch).

This is changed with Windows Vista, all user sessions are non-zero and a console session is equal to a remote desktop session for all intents and purposes.

Every user session on a machine has a unique session ID, and its own instance of CSRSS (Client Server Runtime Subsystem Service) and WINLOGON.
If either of these 2 processes is killed, then the entire user session is killed.
If the "session 0" instance of either of these processes terminates, Windows will reboot - they are essential.

[Paul Adams]

=====================================================================
4. Processes, threads and memory
=====================================================================
Processors, ironically, do not execute processes - they execute threads.
A process can be considered to be a container for a number of threads, so it needs at least 1 thread to actually do anything.

Simple processes can exist with just 1 thread as they do not need any more, more complex applications may create and destroy threads all the time, and run threads on many CPUs at the same time.

A thread is a piece of code with its own "stack" which is a list to indicate which functions in which processes were called so that it can keep a track of where to return to once a call is complete.

A stack follows the principle of "LIFO" (Last In, First Out).
The typical analogy for a stack is to imagine a stack of plates - the plate currently at the top of the stack must be removed first, so the most recently added plate is always the first to come off.

All threads in a process share the same memory space and priority, so a thread that misbehaves could start screwing up life for its sibling threads.

"Multi-tasking" is the ability for an operating system to run threads psuedo-concurrently.
I use the word "pseudo" as for a single core, non-hypthreading processor there can only be 1 active thread at any given time - what the operating system does is to have multiple threads which it switches between, and each gets a little bit of time to run before switching to the next thread.
Every time the processor stops execution of one thread and starts running another, this is called a "context switch".

A running thread may either use its allotted time slice, put itself into a "waiting" state when an I/O operation is pending, or be "pre-emptively" forced back onto the waiting queue by an interrupt.

With a multi-CPU (or multi-core) system there is more than one logical (or physical) processor, so true concurrent processing can take place - 2 distinct threads can be running at the same time.
An application can written to be multi-threaded so that it can take advantage of multiple processors, and Windows itself is a Symmetric Multi-Processor (SMP) operating system which means that it can run its threads on any processor with equal priority.
(The alternative to this is to have the OS run on 1 CPU while its applications run on the others.)

To discuss processes properly we need to explain about how they see memory and also the difference between user mode and kernel mode, first a little aside on naming conventions for storage and throughput...

Gigabytes vs gibibytes
Historically computer designers and programmers have referred to kilobytes, megabytes and gigabytes but not meant their absolutely correct scientific definition.
The reasoning behind this is that computers use base 2 so we don't get convenient round figures often and names were assigned which were close to the correct values.
i.e.
a "kilobyte" officially is 1,000 bytes, but 2^10 is 1,024
a "megabyte" officially is 1,000,000 bytes, but 2^20 is 1,048,576
a "gigabyte" officially is 1,000,000,000 bytes, but 2^30 is 1,073,741,824
a "terabyte" officially is 1,000,000,000,000 bytes, but 2^40 is 1,099,511,627,776

The origins of these scientific prefixes are Greek:
"kilo" is derived from "khilioi", meaning 1,000
"mega" is derived from "megas", meaning "great"
"giga" is derived from "gigas", meaning "giant"
"tera" is derived from "teras", meaning "monster"

There was a big fuss over hard disk manufacturers claiming their products' capacities were N "megabytes" but when reported by an operating system they were shown to apparently have less than this.
It was in the disk manufacturers' interest to make their products have the biggest numbers for sales purposes, so they used the official scientific definition of "mega".
Memory is, as far I am aware, still advertised as N megabytes or gigabytes despite meaning the base 2 definition.

As you can see from the above figures, the larger the unit approximation, the larger the deviation gets - a gigabyte vs a gigabyte could be a difference of over 73,000,000 bytes.

Recent years have shown some people trying to relearn the nomenclature used with storage capacity, though so far I have not seen much evidence of it being widely used - maybe when the US and the UK eventually turn from the imperial measurement system we could see a change

The abbreviations and names for the units have been subtly altered to indicate the difference between base 2 and base 10 values:
kB = kilobyte = 1,000
KiB = kibibyte = 1,024

MB = megabyte = 1,000,000
MiB = mebibyte = 1,048,576

GB = gigabyte = 1,000,000,000
GiB = gibibyte = 1,073,741,824

TB = terabyte = 1,000,000,000,000
TiB = tebibyte = 1,099,511,627,776

I try to refer to "KiB", "MiB" and "GiB" where appropriate (and I remember!), although when talking I will still abbreviate tham to "K", "megs" and "gigs" because I would just feel foolish talking about "kibs, mibs and gibs".

The "bit" and the "byte" are the basic unambiguous definitions.
A "nibble" (4 bits or half a byte) I have seen references to in documents, but never used in practice.
The size of a "word" unfortunately changes depending on the platform being discussed, so is also not a good way to standardise (most commonly 16 or 32 bits, but have been used up to 60 bits).

When discussing bitrates (e.g. audio file sampling rate or theoretical network speed) we use the base 10 definitions, so:
28.8kbps = 28.8kbit/s = 28,800 bits per second
10Mbps = 10Mbit/s = 10,000,000 bits per second

Care should be taken when using anything higher than a byte as confusion can arise.
15 bits, 128 bit/s, 72Kbit/s, 10Mbit/s, 128 bytes, 45 byte/s are all unambiguous, but what about:
48K - is that 48kB (48,000) or 48KiB (49,152)? What about 48k?
88kBps - is that 88 kilobytes per second or 88 kibibytes per second? (capitalised 'B' would imply bytes rather than bits)
12MBps - is that 12 megabytes per second or 12 mebibytes per second? (same with 12MB/s)

A (32-bit) process running on Windows sees, by default, a 4GiB virtual memory space.
The reason for this is that the highest number that can be represented with 32 bits is 2^32=4GiB (4,294,967,296 in decimal, 0xFFFFFFFF in hexadecimal).

The lower memory addresses (0x00000000-0x7FFFFFFF) are for the application itself and represent "user mode".
The upper memory addresses (0x80000000-0xFFFFFFFF) are for private Windows processes & drivers and represent "kernel mode".

Debugging tip:
If tracing a process crash you can spot if a call in the stack or memory access is in user mode or kernel mode by seeing between which of the above boundaries the address falls.

Kernel mode is shared between all user mode applications, because there only needs to be ever 1 copy of it - whereas you could run 1,000 copies of Notepad all with their own unique areas of memory.
If one of those 1,000 instance of Notepad hangs or is terminated, the other 999 remain unaffected.
If there is a problem detected in kernel mode, every single application is affected and so system stability is at risk so Windows takes measures to prevent possible data corruption or loss - you may know this as a bugcheck or "blue screen (of death)".

This addressing system is true regardless of how much memory the system really has - it is what allows Windows programmers to not deal with the trivialities of memory management or system requirements - as far as they are concerned their portion of memory is 2GiB large.
Of course, loading a copy of Notepad does not make Windows try to allocate 2GiB of memory (physical or virtual) - the program can indicate its "working set" preferences, or Windows will just assign pages of memory as the program requires them.

Some applications may want more than 2GiB of memory, and this is where the "/3GB" switch in BOOT.INI comes into play - this moves the split in virtual memory space from 2+2 to 3+1.
User mode gets 3GiB and kernel mode gets just 1GiB - this affects ALL applications, so Windows now has 50% of the memory it had to play with that it had before.
If an application is not written to require or use more than 2GiB then it will get no advantage from this switch (but Windows itself could get a major disadvantage).

To recap, to ensure this point is clear:
An error occurring in user mode is only critical to that application, it cannot crash Windows.
An error occurring in kernel mode is fatal and will cause a bugcheck.

Note: An "error" in this context really means "unhandled exception" - an illegal operation, such as an attempt to read memory outside of the process's owned space. It is possible that data in memory can get corrupted through "errors", but have non-fatal results such as corrupted graphics or sound.

The kernel, besides Windows itself, includes hardware drivers such as disk, graphics and sound, and "filter" drivers such as open file backup agents and anti-virus.
These all share memory and have the potential to corrupt each other which is why great care should be taken when writing code for execution in kernel mode - this is why the most common bugchecks are driver-related and updating drivers can sometimes have a crippling effect.

Debugging tip:
Trying to nail down the cause of bugcheck error codes can be tricky, even if the problem is reproducable (it most often is not).
If the bugcheck is related to memory corruption or leaking ("!analyze -v" in windbg can give you a hint) then you can either:
- use the tool "verifier" to put a watch on all 3rd party drivers
- enable "special pool"

These options will instruct Windows to pay a lot more attention to the drivers or all access to paged and non-paged pool, the bugcheck may change (and become more frequent) as you are now picking up when the corruption occurs rather than when it is detected - so you get a more meaningful memory dump.

Windows has this concept of "virtual memory", where disk space can be used to "page" data or code out of physical memory as needed - this is where PAGEFILE.SYS comes into play.
While some people recommend turning off all swap files on systems with lots of memory, there are a couple of things to take into account:
- some applications will check for a swap file of a certain minimum size, and may refuse to run
- memory dumps when Windows encounters bugchecks require the swap file to be as large as physical memory "plus a bit more" to store the dump temporarily

The working set of a process is the amount of physical memory it is currently using.
Windows can choose to page to disk some or all of the working set of a process if:
- it runs short on free/zeroed pages (empty physical memory)
- the process has been idle for a period of time and the memory manager does some housekeeping to maximise available memory
- the process is minimised and housekeeping kicks in

If a process requests some data which is not in physical memory but has been paged to disk, then a "page fault" is incurred - the system has to temporarily switch to another process to locate and retrieve the data so the original thread can continue.
Despite its name, a page fault is not an error, it is a normal, expected operation.

The amount of virtual memory requested by a process does not necessarily indicate how much it has actually committed, just how much is reserved.
A process which appears to have 200MiB of virtual memory might have nowhere near that much physical or virtual memory used at all, it could be just an estimate based on what the programmer thought it might want to use at peak.

There are also 2 "pools" of memory which are shared across all processes for dynamic use - paged and non-paged.
Paged pool memory can be paged to disk if required, and has an absolute maximum size of 492MiB on Windows 2000, or 650MiB on XP/2003 (the current maximum is calculated at boot time based on the physical memory in the system).
Non-paged pool memory is never paged to disk, so is most commonly used by drivers that cannot incur a page fault during interrupt to request data not in memory - the absolute maximum here is 256MiB.
(The maximums quoted here are based on 32-bit Windows without the /3GB or /USERVA switches used.)

Dynamic Link Libraries (DLLs) are collections of functions which can be called from applications and actually "memory mapped".
If 2 programs refer to the same DLL then they actually use the same instance, it is not loaded twice in memory (unless one of them happens to perform a write operation, in which case a "copy on write" takes place and the DLL is duplicated in read/write mode for this application alone).
One thing to be aware of is that Windows memory management still counts the size of the DLL against every process which refers to it - so in the case of Internet Explorer, for example, the reported memory usage for iexplore.exe is actually less than Task Manager claims as it uses a number of already-loaded system libraries.

[Paul Adams]

=====================================================================
5. Services
=====================================================================
A service is identical to a normal user mode application, with the exception that it can be launched and running without a user being logged on.
(Services also rarely have interfaces, but an application without a user interface is not necessarily a service.)

Services are listed in the registry along with their startup type and dependencies, and they still run in user mode just like any other application.

It is the job of the Service Control Manager (SCM) to start and stop the services (either automatically or on demand) and to take recovery actions specified if a service crashes.

As they are identical to regular user mode applications in other aspects, they have their own protected memory space and sandbox to play in - however, because of dependencies you may find that a service which crashes or fails to start can impact other services.

Through the Control Panel (under Administrative Tools/Services) you can see the services registered, their startup type and current status (started or stopped).
If you view the properties of a service and go to the Dependencies tab you can see the services or system components upon which this service relies, and also the other services which rely on this one.

There are various guides around the Internet describing how to disable "unnecessary" services, but what is correct for one user might be disastrous for another so I dislike explicit "you should stop this service" remarks in general.
View the properties of the Remote Procedure Call (RPC) service and see how many system components rely on it - then decide if it is worth potentially breaking your anti-virus, Windows Update, wireless configuration, etc. services by disabling this one service.

One process which confuses a lot of people is "svchost.exe" - yes there ARE meant to be multiple instances of this process.
This svchost is a special case - its purpose is to collect together a numbers of "not so critical" services to run within 1 process, and the reason for this is to conserve resources.
More critical system services are either independent or run under services.exe ("Services and Controller app").

svchost.exe has the description "Generic Host Proces for Win32 Services".

How does svchost know which services are running in each instance?
Task Manager only shows you "svchost.exe" listed a number of times.

If you run a tool such as Process Explorer from SysInternals you can see a great deal more information on running processes, and if you add the detail column "command line" to the view then you will see each instance of svchost.exe has a unqiue switch after it.
e.g.
C:\WINDOWS\system32\svchost.exe -k DcomLaunch
C:\WINDOWS\system32\svchost.exe -k rpcss
C:\WINDOWS\system32\svchost.exe -k netsvcs
C:\WINDOWS\system32\svchost.exe -k NetworkService

A quick note on naming conventions regarding the registry:
A key (or subkey) is synonymous with a folder in a file system.
A value (or "leaf node") is synonymous with a file in a file system.

A key has only a name - no data - it is just a container for further (sub)keys or values.
A value has a type (e.g. REG_SZ) and associated data (e.g. "this is a string").

People often incorrectly talk about modifying registry keys when editing values.

So how do you know which instance is running which service(s)?
The services are listed as keys under the registry key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services

If we take the Alerter service and view the data for the value "ImagePath" we can see it is:
"%SystemRoot%\system32\svchost.exe -k LocalService"

Now if we look at the WebClient service further down the list, we can see its command line is identical.
So when Windows comes to start one of these services, how does it know which service is starting, and not affect the other?

If we look in the registry under the following key, there are a number of values whose names match the command line arguments of svchost.exe instances:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SvcHost

In the data for each of these values you will see a list of services that all share the same process space.
e.g.
LocalService = Alerter WebClient LmHosts W32Time RemoteRegistry upnphost SSDPSRV WinHttpAutoProxySvc

So here is a list of potential services which would all share the same process, started or stopped individually with what appears to be the same command line.

The slight drawback with this approach is that software firewalls will govern access based on the command line and CRC check of the executable, so any service running under svchost.exe is tarred with the same brush.

Hint:
It is possible to convert practically any executable into a service using the Resource Kit tools instsrv and srvany, as described in KB137890

[Paul Adams]

=====================================================================
6. Drivers
=====================================================================
A driver is something that resides in kernel mode, and as such has to be very carefully written to ensure it does not misbehave, have unhandled exceptions or perform badly.

Windows has a number of drivers available to it natively, but if a new model of device or an entirely new class of device appears after it is released, there is no reasonable way to expect it to be able to communicate with that device.
The manufacturers of the hardware are the origin for the drivers to communicate with their products - if your particular operating system does not have a driver for a device you have installed, then the manufacturer is who you need to contact.

Often, the built-in drivers the manufacturers provide to Microsoft to bundle with the operating system do not contain a lot of the performance-enhancing features that the dedicated (or later versions) of the drivers do.
For this reason you should always ensure you have the latest stable release (ideally WHQL) drivers from the manufacturer for all your hardware.

In some cases manufacturers will use a common chipset, so while Windows does not have a specific driver for a device it might have one that is compatible that works (or at least provides basic functionality).

How does Windows know what a device is?
We are back to this "plug and play" concept - the BIOS has kindly enumerated the devices and presents the hardware ID string (and compatible ID strings) to the OS.
The hardware ID string contains the manufacturer's code plus a code for the device, prefixed with the type of connection (or "bus") the device is on.

e.g. A device on my system has the following hardware ID:
PCI\VEN_10DE&DEV_0092&SUBSYS_060110B0&REV_A1\...

PCI means this is connected to the PCI bus.
VEN_10DE means this is nVidia.
DEV_0092 when used in combination with the vendor ID means this is a 7800GT graphics card.

The device ID is not necessarily unique, this is similar to the fact that a house number is not unique - only when used in combination with the rest of your address.

Hint:
If you get an "unknown device" reported in Device Manager, try using search engines to look for the VEN & DEV strings as a pair.
e.g.
Windows 2000 would probably detected my graphics card as a "Standard VGA Compatible" adapter, as it can determine the class to provide basic functionality, but the 7800GT came out way after SP4 was released.
So if I searched the Internet for "PCI\VEN_10DE&DEV_0092" and "2000" and "driver", I could probably look at the results summary to see what they have in common.

The above describes device drivers - software that handles how the OS can communicate with a physical device inside or attached to the system.
But there is another type of driver, called a filter driver.


A filter driver is something that gets involved with every single I/O request of certain types - the most common filter drivers are backup agents, anti-virus applications and software firewalls.
When you consider the function of anti-virus products, they have to involve themselves with verifying data in memory, on disk or that is transferred across any bus - including the network.
It is actually quite a feat that these products do not have a significant impact on system performance, yet greatly enhance security.

Every filter driver inserts itself into the I/O operation inline, so if you install 4 AV products you will have every operation scanned 4 times and it can have a serious impact on performance (and possibly system stability).
Running more than 1 "real time monitoring" AV product at the same time is not recommended.

Disabling anti-virus is not the same as uninstalling it - the filter driver will still be loaded and operational, just the user mode component is not active - if you are having bugchecks and suspect AV then it should be updated, reinstalled or removed to verify.

A huge percentage of bugchecks in Windows are caused by 3rd party drivers.

[kidzer]

Just read the first 2 posts, but looks an excellent read

[Moby-Dick]

indeed - tis a great guide Paul. I shall be taking some time to study it while I'm knee deep in dead servers tommorow

[ikonia]

awesome, totally awsome.

just the sort of thing that is useful and interesting.

[iranu]

Sticky!

[badass]

Paul Adams=Microsoft Ninja!

[kempez]

Posting to subscribe and say "Good work"

Nice one mate

[kidzer]

Yep, finished reading justnow.

'Awesome' just about sums it up, thoroughly informative and interesting

[Paul Adams]

=====================================================================
7. Debugging
=====================================================================
Debugging is a generic term for the art of finding problems in code, or validating that code is working correctly.

Programmers debug their code while developing applications - they might do this by inserting additional "debug code" to log or display internal actions and data so that what the process is doing can be verified.

Another way to locate bugs is to use a debugger.
Despite the implciation of its name, a debugger does not actually do any kind of bug removal - that's still your job - but what it can be used for is to "wrap" a process so that at any point you can pause it and browse the data, stacks, threads and handles the process might have at a given point in time.

You can also use breakpoints to mark a section of code and let the process run normally until it hits this point, and then pauses it and passes control to the debugger - useful if you know which function is last called when an error occurs but it is difficult or a lengthy process to reproduce.

When a process in Windows tries to do something it should not (most often access memory it does not own or free memory twice), a "first chance" exception is raised.
This is where the OS checks to see if there is any debugger attached to the process - if there is then it is given control and details of the exception.
If there is no debugger attached, or the debugger tells the OS to let the exception pass, then control is passed back to the process to see how it behaves.

If the application is well written, it will have an exception handling routine and can either gracefully deal with it if it is not critical (possibly make an entry in a log ready to send back to the author) or close down in a more friendly manner than crashing.

A lot of applications are not capable of handling exceptions they raise (complexity of code, size of program, time to develop, etc.) and so the exception remains "unhandled".
Windows then checks to see (again) if there is a debugger attached to the process and pass it the "second chance" exception.

If there is no debugger present then the default "post-mortem" debugger kicks in - Dr Watson.

Dr Watson adds an entry to its log and, if the settings are configured so, creates a dump of the process memory space.

The settings for Dr Watson can be seen and changed by running drwtsn32.exe
The information you can see is:
Log file path
Crash dump path and filename
Sound effect to play
Number of instructions to put in the log from the crashed process
Number of crashes to record
Crash dump type (full, mini, "NT compatible")

There are also a number of options to toggle (including whether application crashes create dump files at all) and a summary of the most recent log entries.

So user mode applications can be "live debugged" or have a crash dump analysed to find the root cause - but what about kernel mode exceptions?


We know that exceptions in kernel mode cause bugchecks to protect the integrity of data, but the kernel consists of many processes, threads, handles, objects, memory pools, stacks and they can corrupt each other - so how to "debug" the entire kernel?

If we run a debugger as a user mode process and try to freeze the kernel, we have just frozen the entire OS - so the debugger would freeze too...

Recall I mentioned the "/DEBUG" BOOT.INI switch earlier - with this option enabled (along with a DEBUGPORT to use), we can attach an external debugger running on another machine to the system and debug Windows itself.

Now you can manually break into a running system, freeze it and analyse all the kernel mode data structures, currently running process and threads, memory limits and use - and the system to be debugged does not even have any screen updates and the mouse does not even move.

If you want, at any time you can tell the debugger to release and let the debugged system continue on its way - other than a clock change it won't notice any difference.

If you have a debugger attached when kernel mode execption occurs, the debugger is notified and given the opportunity to see the system in its broken state without (or before) creating a memory dump file.

This is a live kernel debug, and we have the same post-mortem debug option by loading the memory.dmp file created when Windows bugchecks into the debugger.
However, if the dump options have been disabled then there is nothing to analyse.

If the dump option is set to "mini dump" then a very tiny amount of data is stored.

If a "kernel mode dump" is selected, then the code and data stored in physical memory for the kernel is dumped to a file - this is the most common dump file that actually contains useful data, but the size of the dump cannot be known beforehand as the amount of physical memory used by kernels will vary.
For a kernel dump file to be created a swap file is required, and there are minimum page file sizes dependent on the amount of physical memory installed:
<128MiB -> 50Mib swap file
128MiB 4GiB -> 200MiB swap file
4GiB-8GiB -> 400MiB swap file
8GiB+ -> 800MiB swap file
The only way to guarantee a kernel dump can be stored would be to set the page file size at 2GiB+1MiB, as on a 32-bit system the kernel cannot be larger than 2GiB (and there is a little overhead for the dump file header).
There must also be at enough free disk space equal to the size of the dump created on the system volume and the volume where memory.dmp is specified.

If a "complete memory dump" is selected, then all physical memory is dumped to a file, and the swap file and free disk space on the system volume must be at least as big as physical memory plus 1MiB each to guarantee a working dump.

The reason for the swap file being required for kernel or complete memory dumps is that this is where the memory is dumped to initially, and the reason for the free disk space requirement is that this dump is then copied to memory.dmp before the swap file is cleared and the system restarted.

A complete dump is not often necessary as bugchecks cannot be caused by user mode processes.


A bugcheck can also be instigated manually by setting the registry value CrashOnCtrlScroll to 1 and using a key combination on a (non-USB) locally-connected keyboard.

Why would you want to bluescreen your machine on purpose?
In the event where you have a memory leak in kernel mode, most likely.
Memory leaks do not generate bugchecks - a poorly written kernel mode process might just consume and consume and consume until there is nothing left for other Windows processes and no new processes can spawn, deadlocking the system but not actually causing an exception.

In this situation (where you most likely have event IDs 2019 and 2020 logged), you could manually crash the system to produce a memory dump, and then analyse it to see which process consumed all the memory.


User mode application crashes are the responsibility of the vendor (remember that due to the "extension" model of Explorer and Internet Explorer, 3rd party plugins can be the cause of crashes in core components).

Kernel mode exceptions are invariably the result of a problem with a 3rd party driver (device or filter).

In either case, the first check would be to ensure the latest version of the software is installed (the likely faulting component being identified by analysing the dump).
If the software is up to date, or the problem persists, then remove the software entirely and see if it still occurs (not always practical in the case of storage or display drivers).
If all the software is up to date and no individual piece of software can be identified as the cause, then a live debug may be required.

[Dav0s]

http://dictionary.reference.com/search?r=2&q=brief


very very nice mate.