Tags: Windows »»»» Open Source
Microsoft's Windows operating system family is an egregious example of closed source operating systems and the risk to consumers from overly-controlling manufacturers. Many of us worry what Microsoft does with information collected from its customers, and chafe at the lack of control we have over our computers. But the popularity of Windows means there's lots of enticing software and a large number of folks comfortable in that environment. As they might wish for an open alternative, with a strong NO SPYWARE guarantee, they want a thing which behaves like Windows. Getting there means undertaking an effort to reverse engineer a Windows implementation in the open source world.
The phrase reverse engineered, a.k.a. a clean room implementation, refers to handing software engineers the API documentation, and other published manuals, and telling them to implement the same thing. They cannot examine source code, but they could have a running machine on which to test software. It would be legally problematic to dissassemble the software on that running machine because of the risk of leaking code/implementation into their supposedly reverse engineered implementation. Ideally the reverse engineered implementation is completely new code that happens to behave the same as the original.
In other words - a team of software engineers would have to implement everything which makes up Windows, from the hardware abstraction layer on up to user interface libraries, and even some of the applications.
Before getting to the question of whether any Windows OS was reverse engineered we need to cover some basic terminology. If you skip down you'll find a few projects which reverse engineered certain aspects of Windows.
Definitions
To understand what it would mean to reverse-engineer Windows, we have to pick apart what that would mean.
The "Windows API" is the primary interface between an application and the Windows operating system. More formerly: The Windows API, informally WinAPI, is Microsoft's core set of application programming interfaces (APIs) available in the Microsoft Windows operating systems. There are several subsystems to these API's:
- Win16 - the original 16-bit programming interface primarily for Windows implementations that ran on DOS.
- Win32 - The 32-bit programming interface introduced with Windows NT. There was a Win32c library ('c' for 'compatibility') shipped in Windows 95
- Win32s - A subset of Win32 which was implemented for Windows 3.1
- Win64 - The 64-bit programming interface for 64-bit systems
- WinCE - This is where Microsoft proves its marketing department is clueless (look at the letters, then wince). Anyway, CE meant Compact Edition or some such, and this was implemented for the Windows CE operating system which ran on early mobile devices.
Then there are other user interface libraries that run on top of the Windows API
- Direct2D / Direct3D - Graphics support for 2D/3D
- DirectSound - Audio support
- DirectInput - Input devices
- DirectPlay - Gaming support
- DirectDraw - High-speed 2D graphics
- DirectShow / Media Foundation - Multimedia and video playback
That barely touches the surface of various API libraries supplied by Microsoft. [Wikipedia1]
Microsoft bundles a long list of components, services, and applications into every Windows release. [Wikipedia2]
And then there is the interior structure of Windows NT. The API's interface with the underlying operating system, and it may be necessary to reverse-engineer some/all of that. Even this image is incomplete given all the file systems (for example) supported by Windows. [Wikipedia3]
IBM's OS/2
There was a compatibility layer in OS/2 allowing Win16 applications to run. However, given that IBM and Microsoft partnered on OS/2 implementation, it's obvious that Microsoft will have suppliled the required code.
Mainsoft - implemented Win32 API's
A company called Mainsoft (I worked for them from 1994 - 1996) developed MainWin, a commercial tool which implemented the WIN32 API and which allowed other toolkits to run on top of the library, like MFC. It allowed folks to recompile their Windows applications on Unix for display on X11 displays.
For MFC support (and other libraries), we simply took the library source code from Visual Studio and recompiled them, as-is with no changes, using the MainWin library.
Initially MainWin was a reverse engineered implementation of the WIN32 API. It was developed by two French software engineers, Jose Luu and Gregory Popovich, who brought their company, Mainsoft, to Silicon Valley because of the venture capital money. By the time I started at Mainsoft, the office in Sunnyvale had about 20 employees and a growing client base.
Around the time I joined the company paid for a Windows License. At that time there was some legal concerns about Microsoft's closed source practices, and Microsoft entered into source licensing agreements with several companies. This meant we had source code for both Windows 95 and Windows NT (v3.51 and the beta of v4) sitting on our file server.
Eventually we paid for a Windows license, under the Windows Interface Source Environment program[Wikipedia3], and had the Win95 and WinNT4 source code in hand. IIRC this occurred in 1995, and as I recall there was some political gamesmanship going on in the computer industry at the time. For example the lawsuits related to Microsoft's requirement imposed on PC manufacturers to ONLY supply Windows, dissallowing PC makers from shipping other operating systems. A part of Microsoft's response was a program for licensing Windows source code, which Mainsoft took advantage of.
At first we only used the Windows source for reference, preferring to keep the MainWin source tree clean of Microsoft source code. However, while fixing some bugs in the Listbox control I found that by directly incorporating source from the Windows source tree it was possible to fix a much larger list of bugs. That resulted in the team going through a larger-scale import of Windows source code into MainWin.
At another stage of the company we worked on porting Internet Explorer to Unix -- apparently at the time Microsoft thought they were losing $1 million per day by not having a web browser on Unix. Eventually that led to a joint project with Microsoft engineers to -- I only know this 3rd hand since I'd left the company by this point -- to not only port Internet Explorer, but to import large quantities of Windows source into MainWin.
There was another company, Bristol Technology, in the same time frame against which we were in competition.
Bottom line - MainWin did not reverse engineer the entire Windows operating system. It started as a reverse-engineered implementation of parts of the Win32 API. Eventually Windows source code was imported into the product. MainWin allowed application developers to recompile Win32-based applications on Unix/X11 systems.
Bristol Technology
This was a technology company in the early 1990's which was Mainsoft's primary competitor. They also had developed a library allowing folks to port Windows applications to Unix.
I had an interesting competition win against that company. A company in Phoenix had bought Bristol's product and attempted to port their application to run on Unix with that product. But, they failed, and called us to see if we could do any better. I happened to be in Phoenix on another assignment, and then spent a couple days in the office of this new customer. Within a few hours I had their software running on MainWin, where Bristol had failed after months of effort. As a happy customer they GAVE me their copy of Bristol's product, which I took back to the office as a trophy.
Bristol had a stake in the competition to port Internet Explorer to Unix. In early 1995 Mainsoft's President learned of the porting project and that Bristol was the sole participant in that project. He got us involved, and we put in a lot of effort, only to fail and see Bristol win the deal. But that win was relatively short-lived since, as I said earlier, Mainsoft was eventually chosen (after I left) to work with Microsoft on porting Internet Explorer to Unix. The Unix version of Internet Explorer which was finally delivered to the public used the Mainsoft library, rather than Bristol's.
Wine
Wine[Wikipedia4] - which runs on Linux and Mac OS X - is a compatibility layer allowing for Windows applications to run on the aforesaid operating systems. Wine does this by translating Windows API functions into POSIX-compatible functions. Wine supports a Windows-compatible directory structure, and alternate implementations of a long list of Windows services and components.
The result is that in many cases you can simply install a Windows application, and it will behave right. In other cases, unfortunately, the application will crash, because Wine is still a work in progress.
The name is a recursive backronym meaning Wine is not an emulator. Wine does not emulate Windows, it instead is a compatibility layer.
Some Windows application vendors support usage on non-Windows systems by promising their application will run under Wine. For Enterprise Architect the system requirements list this:
- Linux
- Linux Operating System (kernel 2.4 or later)
- Wine 1.8 (minimum), 1.9x or later (recommended) (Or CrossOver Linux 15.0.1, or later)
- Microsoft Data Access Components (MDAC) 2.8
- 2GB of RAM
- 300MB of available hard-disk space
- 1280*720 or higher resolution display
- Mac OS X
- Mac OS 10.8 or later
- Wine 1.8 (minimum) (Or CrossOver Mac 15.0.1, or later)
- Microsoft Data Access Components (MDAC) 2.8
- Intel® based processor
- 2GB of RAM (128 MB or higher recommended)
- 300MB of available hard-disk space
- 1280*720 or higher resolution display
Under Wine, applications are installed as-is rather than recompiled.
Bottom line - While incredibly useful, Wine does not implement the entirety of Windows, and it is still very much an unfinished thing.
ReactOS -- moving towards a full Windows implementation
There is a thing called ReactOS[Wikipedia5] that goes even further. The goal is to implement a binary-compatible operating system that behaves identially with Windows in every way.
Going by our earlier review of ReactOS (Quick review of ReactOS, a Windows alternative with blast-from-the-past user experience), while it is impressive it is not ready for prime time.
It reuses parts of the Wine project to implement the GUI layers.
It behaves very much like WindowsNT sometime after WinXP.
Samba - the SMB server and client for Linux, Mac OS X, FreeBSD and other systems
Samba is both server and client components implementing the Windows SMB and CIFS protocols. SMB and CIFS was formerly a Windows-only thing, but the Samba team went ahead and reverse-engineered the protocol.
Because of the Samba project, SMB/CIFS support is available for pretty much every operating system and NAS device and even devices like mobile phones.
Apparently Microsoft was initially unhappy about this, but eventually came around.
Mono - open source implementation of .NET
Initially the Mono project was a reverse engineered open source project to implement .NET technologies.
Apparently the heavens moved or something because the
Mono Project home page now says:
Sponsored by Microsoft, Mono is an open source implementation of Microsoft's .NET Framework based on the ECMA standards for C# and the Common Language Runtime. A growing family of solutions and an active and enthusiastic contributing community is helping position Mono to become the leading choice for development of cross platform applications.
