Was the Windows operating system ever reverse engineered?

By: (plus.google.com) +David Herron; Date: March 22, 2018

Tags: Windows » Open Source

Microsoft's Windows operating system family is an egregious example of closed source operating systems and the risk to consumers from overly-controlling manufacturers. Many of us worry what Microsoft does with information collected from its customers, and chafe at the lack of control we have over our computers. But the popularity of Windows means there's lots of enticing software and a large number of folks comfortable in that environment. As they might wish for an open alternative, with a strong NO SPYWARE guarantee, they want a thing which behaves like Windows. Getting there means undertaking an effort to reverse engineer a Windows implementation in the open source world.

The phrase reverse engineered, a.k.a. a clean room implementation, refers to handing software engineers the API documentation, and other published manuals, and telling them to implement the same thing. They cannot examine source code, but they could have a running machine on which to test software. It would be legally problematic to dissassemble the software on that running machine because of the risk of leaking code/implementation into their supposedly reverse engineered implementation. Ideally the reverse engineered implementation is completely new code that happens to behave the same as the original.

In other words - a team of software engineers would have to implement everything which makes up Windows, from the hardware abstraction layer on up to user interface libraries, and even some of the applications.

Definitions

To understand what it would mean to reverse-engineer Windows, we have to pick apart what that would mean.

The "Windows API" is the primary interface between an application and the Windows operating system. More formerly: The Windows API, informally WinAPI, is Microsoft's core set of application programming interfaces (APIs) available in the Microsoft Windows operating systems. There are several subsystems to these API's:

  • Win16 - the original 16-bit programming interface primarily for Windows implementations that ran on DOS.
  • Win32 - The 32-bit programming interface introduced with Windows NT. There was a Win32c library ('c' for 'compatibility') shipped in Windows 95
  • Win32s - A subset of Win32 which was implemented for Windows 3.1
  • Win64 - The 64-bit programming interface for 64-bit systems
  • WinCE - This is where Microsoft proves its marketing department is clueless (look at the letters, then wince). Anyway, CE meant Compact Edition or some such, and this was implemented for the Windows CE operating system which ran on early mobile devices.

Then there are other user interface libraries that run on top of the Windows API

  • Direct2D / Direct3D - Graphics support for 2D/3D
  • DirectSound - Audio support
  • DirectInput - Input devices
  • DirectPlay - Gaming support
  • DirectDraw - High-speed 2D graphics
  • DirectShow / Media Foundation - Multimedia and video playback

That barely touches the surface of various API libraries supplied by Microsoft. [Wikipedia1]

Microsoft bundles a long list of components, services, and applications into every Windows release. [Wikipedia2]

Microsoft Windows NT architecture from - By The original uploader was Grm wnr at English Wikipedia Later versions were uploaded by Xyzzy n at en.wikipedia. - Transferred from en.wikipedia to Commons., CC BY-SA 3.0, `https://commons.wikimedia.org/w/index.php?curid=2242280`

And then there is the interior structure of Windows NT. The API's interface with the underlying operating system, and it may be necessary to reverse-engineer some/all of that. Even this image is incomplete given all the file systems (for example) supported by Windows. [Wikipedia3]

IBM's OS/2

There was a compatibility layer in OS/2 allowing Win16 applications to run. However, given that IBM and Microsoft partnered on OS/2 implementation, it's obvious that Microsoft will have suppliled the required code.

Mainsoft - implemented Win32 API's

A company called Mainsoft (I worked for them from 1994 - 1996) developed MainWin, a commercial tool which implemented the WIN32 API and which allowed other toolkits to run on top of the library, like MFC. It allowed folks to recompile their Windows applications on Unix for display on X11 displays.

For MFC support (and other libraries), we simply took the library source code from Visual Studio and recompiled them, as-is with no changes, using the MainWin library.

Initially MainWin was a reverse engineered implementation of the WIN32 API. We had zero access to Microsoft source code, and our team did a clean reverse engineering of a large portion of Win32 and Win16 functionality.

Eventually we paid for a Windows license, under the Windows Interface Source Environment program[Wikipedia3], and had the Win95 and WinNT4 source code in hand. IIRC this occurred in 1995, and as I recall there was some political gamesmanship going on in the computer industry at the time. For example the lawsuits related to Microsoft's requirement imposed on PC manufacturers to ONLY supply Windows, dissallowing PC makers from shipping other operating systems. A part of Microsoft's response was a program for licensing Windows source code, which Mainsoft took advantage of.

At first we only used the Windows source for reference, preferring to keep the MainWin source tree clean of Microsoft source code. However, while fixing some bugs in the Listbox control I found that by directly incorporating source from the Windows source tree it was possible to fix a much larger list of bugs. That resulted in the team going through a larger-scale import of Windows source code into MainWin.

At another stage of the company we worked on porting Internet Explorer to Unix -- apparently at the time Microsoft thought they were losing $1 million per day by not having a web browser on Unix. Eventually that led to a joint project with Microsoft engineers to -- I only know this 3rd hand since I'd left the company by this point -- to not only port Internet Explorer, but to import large quantities of Windows source into MainWin.

There was another company, Bristol Technology, in the same time frame against which we were in competition.

Bottom line - MainWin did not reverse engineer the entire Windows operating system. It started as a reverse-engineered implementation of parts of the Win32 API. Eventually Windows source code was imported into the product. MainWin allowed application developers to recompile Win32-based applications on Unix/X11 systems.

Wine

Wine[Wikipedia4] - which runs on Linux and Mac OS X - is a compatibility layer allowing for Windows applications to run on the aforesaid operating systems. Wine does this by translating Windows API functions into POSIX-compatible functions. Wine supports a Windows-compatible directory structure, and alternate implementations of a long list of Windows services and components.

The result is that in many cases you can simply install a Windows application, and it will behave right. In other cases, unfortunately, the application will crash, because Wine is still a work in progress.

The name is a recursive backronym meaning Wine is not an emulator. Wine does not emulate Windows, it instead is a compatibility layer.

Some Windows application vendors support usage on non-Windows systems by promising their application will run under Wine. For Enterprise Architect the system requirements list this:

  • Linux
    • Linux Operating System (kernel 2.4 or later)
    • Wine 1.8 (minimum), 1.9x or later (recommended) (Or CrossOver Linux 15.0.1, or later)
    • Microsoft Data Access Components (MDAC) 2.8
    • 2GB of RAM
    • 300MB of available hard-disk space
    • 1280*720 or higher resolution display
  • Mac OS X
    • Mac OS 10.8 or later
    • Wine 1.8 (minimum) (Or CrossOver Mac 15.0.1, or later)
    • Microsoft Data Access Components (MDAC) 2.8
    • Intel® based processor
    • 2GB of RAM (128 MB or higher recommended)
    • 300MB of available hard-disk space
    • 1280*720 or higher resolution display

Under Wine, applications are installed as-is rather than recompiled.

Bottom line - While incredibly useful, Wine does not implement the entirety of Windows, and it is still very much an unfinished thing.

ReactOS -- moving towards a full Windows implementation

There is a thing called ReactOS[Wikipedia5] that goes even further. The goal is to implement a binary-compatible operating system that behaves identially with Windows in every way.

Going by our earlier review of ReactOS (Quick review of ReactOS, a Windows alternative with blast-from-the-past user experience), while it is impressive it is not ready for prime time.

It reuses parts of the Wine project to implent the GUI layers.

It behaves very much like WindowsNT sometime after WinXP.

« What did computers use before the Windows operating system was invented? Facebook, OAuth authorization protocol, user responsibility, Facebook responsibility »
2016 Election Acer C720 Ad block AkashaCMS Amazon Amazon Kindle Amazon Web Services America Amiga and Jon Pertwee Android Anti-Fascism AntiVirus Software Apple Apple Hardware History Apple iPhone Apple iPhone Hardware April 1st Arduino ARM Compilation Artificial Intelligence Astronomy Astrophotography Asynchronous Programming Authoritarianism Automated Social Posting AWS DynamoDB AWS Lambda Ayo.JS Bells Law Big Brother Big Data Big Finish Big Science Bitcoin Mining Black Holes Blade Runner Blockchain Blogger Blogging Books Botnets Cassette Tapes Cellphones China China Manufacturing Christopher Eccleston Chrome Chrome Apps Chromebook Chromebox ChromeOS CIA CitiCards Citizen Journalism Civil Liberties Clinton Cluster Computing Command Line Tools Comment Systems Computer Accessories Computer Hardware Computer Repair Computers Conservatives Cross Compilation Crouton Cryptocurrency Curiosity Rover Currencies Cyber Security Cybermen Cybersecurity Daleks Darth Vader Data backup Data Formats Data Storage Database Database Backup Databases David Tenant DDoS Botnet Department of Defense Department of Justice Detect Adblocker Developers Editors Digital Photography Diskless Booting Disqus DIY DIY Repair DNP3 Do it yourself Docker Docker MAMP Docker Swarm Doctor Who Doctor Who Paradox Doctor Who Review Drobo Drupal Drupal Themes DVD E-Books E-Readers Early Computers Election Hacks Electric Bicycles Electric Vehicles Electron Eliminating Jobs for Human Emdebian Encabulators Energy Efficiency Enterprise Node EPUB ESP8266 Ethical Curation Eurovision Event Driven Asynchronous Express Face Recognition Facebook Fake News Fedora VirtualBox Fifth Doctor File transfer without iTunes FireFly Flash Flickr Fraud Freedom of Speech Front-end Development G Suite Gallifrey git Github GitKraken Gitlab GMAIL Google Google Chrome Google Gnome Google+ Government Spying Great Britain Green Transportation Hate Speech Heat Loss Hibernate Hoax Science Home Automation HTTP Security HTTPS Human ID I2C Protocol Image Analysis Image Conversion Image Processing ImageMagick In-memory Computing InfluxDB Infrared Thermometers Insulation Internet Internet Advertising Internet Law Internet of Things Internet Policy Internet Privacy iOS Devices iPad iPhone iPhone hacking Iron Man iShowU Audio Capture iTunes Janet Fielding Java JavaFX JavaScript JavaScript Injection JDBC John Simms Journalism Joyent Kaspersky Labs Kext Kindle Kindle Marketplace Large Hadron Collider Lets Encrypt LibreOffice Linux Linux Hints Linux Single Board Computers Logging Mac Mini Mac OS Mac OS X Machine Learning Machine Readable ID macOS macOS High Sierra macOS Kext MacOS X setup Make Money Online March For Our Lives MariaDB Mars Mass Violence Matt Lucas MEADS Anti-Missile Mercurial MERN Stack Michele Gomez Micro Apartments Microsoft Military AI Military Hardware Minification Minimized CSS Minimized HTML Minimized JavaScript Missy Mobile Applications Mobile Computers MODBUS Mondas Monetary System MongoDB Mongoose Monty Python MQTT Music Player Music Streaming MySQL NanoPi Nardole NASA Net Neutrality Network Attached Storage Node Web Development Node.js Node.js Database Node.js Performance Node.js Testing Node.JS Web Development Node.x North Korea npm NVIDIA NY Times Online advertising Online Community Online Fraud Online Journalism Online Photography Online Video Open Media Vault Open Source Open Source and Patents Open Source Governance Open Source Licenses Open Source Software OpenAPI OpenJDK OpenVPN Palmtop PDA Patrick Troughton Paywalls Personal Flight Peter Capaldi Peter Davison Phishing Photography PHP Plex Plex Media Server Political Protest Politics Postal Service Power Control President Trump Privacy Production use Public Violence Raspberry Pi Raspberry Pi 3 Raspberry Pi Zero ReactJS Recaptcha Recycling Refurbished Computers Remote Desktop Removable Storage Republicans Retro Computing Retro-Technology Reviews RFID Rich Internet Applications Right to Repair River Song Robotics Robots Rocket Ships RSS News Readers rsync Russia Russia Troll Factory Russian Hacking Rust SCADA Scheme Science Fiction SD Cards Search Engine Ranking Season 1 Season 10 Season 11 Security Security Cameras Server-side JavaScript Serverless Framework Servers Shell Scripts Silence Simsimi Skype SmugMug Social Media Social Media Networks Social Media Warfare Social Network Management Social Networks Software Development Software Patents Space Flight Space Ship Reuse Space Ships SpaceX Spear Phishing Spring Spring Boot Spy Satellites SQLite3 SSD Drives SSD upgrade SSH SSH Key SSL Stand For Truth Strange Parts Swagger Synchronizing Files Tegan Jovanka Telescopes Terrorism The Cybermen The Daleks The Master Time-Series Database Tom Baker Torchwood Total Information Awareness Trump Trump Administration Trump Campaign Twitter Ubuntu Udemy UDOO US Department of Defense Virtual Private Networks VirtualBox VLC VNC VOIP Vue.js Walmart Weapons Systems Web Applications Web Developer Resources Web Development Web Development Tools Web Marketing Webpack Website Advertising Weeping Angels WhatsApp William Hartnell Window Insulation Windows Windows Alternatives Wordpress World Wide Web Yahoo YouTube YouTube Monetization