Tag Archive : c++

/ c++

Mixing C++ with AMD64 (x86_64) assembly

9 November 2019 | asm, C++, cmake | No Comments

Lately, I’ve been dabbling into some “closer to the metal” kind of programming.

On most compilers (Visual Studio’s one for instance) it used to be rather easy to mix assembly code and C++ code using a feature called inline assembly where the ASM code will be put in a block (decorated with a special macro/symbol like _asm for instance), and when the compiler sees that, it will put the content of this “as is” inside of the compiled code.

Well, that feature don’t exist once you compile in 64bit mode, and also, mixing programming languages in a single file is a bit gross. Each time I see this, I think it feel rather ugly…

Instead, it would be good to separate that code into files, that will be assembled into object code you can link to your program as you do with C++ compilation units and static libraries…

So, how do you go about doing this, and doing it in a simple, and cross platform manner.

Well, first of all, there’s 2 things to consider when writing code in assembly.

1) The code you write obviously depend on the machine instruction set, since you are basically asking a CPU to do work, but you do it *instruction per instruction*. This means that you can only target one family of processors with that code. So when I say cross platform, I mean different rutnime on compatible machines. For example Windows VS Linux VS MacOs running on good old Intel (or compatible) chips.

2) Generally, you don’t just write instructions when writing assembly, you also write directive for the program that turn your code from human readable thing with letters and spaces and comments into the digital gibberish the CPU actually eats up to work. And there are multiple of them, and they aren’t really compatible with each other.

I was on a windows machine when I started experimenting, and saw that there was obviously the MASM option, the macro assembler from Microsoft that was already installed on my box, as it is distributed with Visual Studio. Great… And I started playing with it, it’s not hard. I even got CMake to generate automatically a build system that work. But sadly these effort were vain as I quickly released that, I would not be able to use Microsoft’s assembler on Linux or a Mac for quite obvious reasons

So, as always… Open-Source software win the day!

I was working on something totally unrelated that involved me having to build from source the OpenSSL library. When looking at the dependencies I needed to install on a bog standard windows box to do so, beside their horrendous build system written in Perl, one program stood up: NASM.

NASM is, my friend Google told me, the Netwide Assembler. It’s a BSD licensed Open-Source macro-assembler for 16, 32 and 64bit intel chips! Great news! It is cross platform and work on Linux and MacOS too (and I bet a bit more stuff!)

Also, I started googling around to see if CMake could auto-magically handle it, and indeed it does since a prehistoric version.

So, I got around to test that, and well, beside a tiny snag on MacOS that is apparently due to the way MachO object files differ from ELF files (a sad story of respecting standards about leading underscores) that is quickly fixed by telling NASM to prefix all global symbols with a _ character, I got a thing that just worked.

So, the result of this silly experiment is a demonstration repository that has nothing impressive to show beside how easy it works. It’s here, there’s C++ code, there’s ASM code, there’s a CMake project file that compiles everything together, the C++ code declare some extern "C" functions with names and interfaces that just happen to match the globally defined exportable symbols in the assembly code, and… voilà!

https://github.com/Ybalrid/cmake-cpp-nasm

Feel free to steal the literal 4 lines of configuration needed for this thing to work and go do some cool stuff! 😀

SDL: not (just) a 2D graphics library

3 September 2019 | C++, cross platform, SDL | No Comments

SDL logo

My first real introduction with programming was with the C programming language, on a French website that was, at the time, known by the “Le Site Du Zéro” (think “The newbie’s website”).

At this time, I was in middle school, and I was pretty bored by it. It’s around that time that I started to have really access to the internet at home, and started spending a lot of time on it.

There were a few things behind this, mostly I was looking for trying to understand how video games works and how they were made. I was also trying to learn more about how computers work, without actually knowing what to look for, and I started dabbling into Linux in that time. This was around 2007.

This website had a well written, comprehensive, and actually fun “Introduction to C programming” tutorial. (They called them tutorials, but think of them more like courses…) Today, this website became “Open Classrooms”, a popular french MOOC platform, but the originals “tutos” are still up, and were (for some of them) improved, some turned into video courses.

This one was divided into 3 parts, one that is you “C programming 101” type of stuff, were they show you how to create variables, functions, if statements, loops, structures… This kind of things, the 2nd one is about more advanced things with pointers, memory allocations, arrays and whatnot, and the 3rd one is about showing you how to use a library, and takes example of the SDL to create a window, display things, and make a little sokoban game (with stolen Super Mario World sprites 😉).

In this thing, SDL is described as being a 2D graphics library for games, and for years, I regarded the SDL as being just that. I have no clue if other people had this misconception, but when I look for things about the SDL, I mostly see things about making 2D games with it, or seeing it merely as a way to get a window and a screen and not much else.

But, oh man. It’s so much more than that.

The SDL name did make little sense to me at that time, but I never really thought about it. SDL is the Simple Directmedia Layer. A better way of describing the SDL would be a “cross platform API for doing most of the things you would want to do with an OS, including drawing windows, creating threads, loading DLLs, and pretty much anything.

I’ve learned recently that it was developed to be used as an aid for porting software from windows to other platform. The main idea being to support functionalities in a convenient package that are analogue to what was available with Microsoft DirectX. This was developed by former Loki Software employee Sam Lantinga. The business-model of that company was to port video games from windows to other platforms, and mostly Linux.

The SDL works on pretty much anything you would want to use. Today, for me it looks more like a fundamental building block for multimedia programs (video games, and other interactive things with custom graphics), and without having to worry too much about the platform you are running into.

If you use the SDL, anything you do through it will work on a computer running Windows, MacOS, Linux, flavors of BSD, obscure little things like Haiku (Haiku is awesome. This is a really interesting operating system project), any kind of smartphone being Android or iOs, and probably much more things like video game consoles. All these platforms are available to you without having to change your code too much (not at all most of the time).

Since version 2.0 came out, the SDL is licensed under the zLib license. Before that, SDL was LGPL. LGPL is not a problem in commercial, closed source software (like video games usually are) when the library is dynamically linked to the program (for example, on Windows, the library is a .DLL file distributed alongside the binaries of the game.) This permit you to recompile and change the library, and the developers of the game needs to make the source code of the version used with the game available to you, including any changes they may have done to it, and under the same rights. This can make some company’s lawyers uneasy. the zLib license is a much shorter, much straightforward license that pretty much says “do whatever with the code of this, and credit about it is appreciated”

Also since version 2.0, the huge feature list of the library was expended further. The impressive feature list can be read here.

I spent quite some time misrepresenting what the SDL was, thinking that it was just a lousy 2D graphics libraries with a software renderer, and not the impressive tool that it actually is.

Few years ago, while searching content from Vavle about their SteamVR system, I stumbled across this talk by Ryan C. Gordon, a prolific freelance programmer that, if you ever ran a video game natively on Linux, there’s a pretty high chance he helped make this a reality. I really invite you to watch this, and appreciate all the hard work this simple C library can do for you in just a few lines…

As a side note, I really should plug a little project a good friend of mine started some time ago (and which I contribute to, because it’s awesome) to make a safe, C++17, RAII wrapper around most of the useful bits of SDL for 2D/OpenGL/Vulkan programming. Check it out here: https://github.com/Edhebi/cpp-sdl2

Sometimes I wonder why some things are inside the C and C++ standard libraries, and some aren’t.

As far as I can be bothered to read the actual “standards” document (that are mostly written in legalise, not in understandable english for you and me). Theses languages are defined against an “abstract machine”, and the actual real-world implementation of them, on computers that actually exists, should follow the behavior described for that thing, modulo some implementations details.

Beside the specific case of having theses languages in a “free standing environment” (meaning that the code written isn’t actually relying on being executed inside an operating system, but is directly running on the bare hardware), It seems that some notions, like the OS having a “filesystem” where textual paths can points to files that can be opened, read and modified, is pretty standard.

What is strange about that is, if the notion of files and file systems are part of the standard library for both C and C++, (and were present since the beginning) networking sockets doesn’t exist in both languages. And C++ gained the notion of creating multiple threads of execution, and manipulating them inside it’s standard library only in 2011.

All of theses concepts : files, threads, and sockets, are Operating System specific constructs. Opening a file on Linux is fairly different that opening a file on Windows for example. But the standard library offer a single, unique, and portable way of doing so.

These three things are present in all operating systems in use for the past decades now (since the 70’s at least?). I find it strange that the C standard library only includes files. Since C++ now also has threads, I will consider this a non-issue. So let’s talk about the other one…

A song of files and sockets

I would like to take some time to discuss writing some low-level network code in C++. The current interface to get data to and from a network we know today are using a notion called sockets.

For lack of a better analogy, a socket can be thought as some kind of “magical file” that, when, written into, will send the bytes that has been written to on the network, and when receiving bytes, they will be accessible by reading said file. This notion comes from the UNIX world, where everything is effectively a file. And nothing is wrong with that, it’s actually a really simple, straightforward way of doing things.

Most of the operating systems that matters today are using this analogy. When I say OSes that matters today, I’m thinking of both the modern UNIX derivatives (Linux, macOS, and the rest of the BSD family). And Microsoft Windows.

The Windows socket API has been mostly borrowed from BSD anyway, if you remove some oddities like a few renamed functions, a few changed data types, and the added initialization procedure, Windows sockets are equivalent of sockets you have on Linux.

100% non-standard code

But, none of this exist inside the standard library of these languages. When you are doing socket programming on Linux, you are not calling functions of the library, you are performing Linux Kernel System calls, and you are dealing with file descriptors and bytes.

On Windows, you are calling part of the Win32 API (called WSA for Windows Socket API). This situation is unfortunate, because it means that I, as a C++ programmer, I need to make sure that my code will work both under Linux and under Windows. There’s no one single networking API that I can use everywhere without thinking about it. Sure, it’s 80% similar, or maybe 90% similar, but still, if I need to put #ifdef WIN32 in my code for something as fundamental as sending bytes to another computer in a network in 2018, we are doing something wrong.

Moreover, all theses OS-level API are implemented in C. Not C++. This means that everything you have are functions and structures. When describing socket configuration, you are filling structures and passing them to functions. When you have to reference a specific socket, you need to keep a little token and give it to a functions. When you need to read data, you need to have a buffer with the correct amount of bytes ready and give a pointer, alongside a variable containing the size of said buffer to a function, and make sure that you don’t mix them together.

Basically, you are doing 1970’s level computer science. This is fine for the lowest level code out there, but not for the code of an application.

Unnecessary added complexity

There are solutions to solve this, and some that are even well advanced in the path of getting standardized inside the C++ language, like Boost.Asio. But, In my humble opinion, there’s a fundamental problem with Asio itself: it is Asio.

For those who aren’t familiar with Asio, it’s name stands for “asynchronous input and output”. It’s a library, a good library for what it’s name stands for: doing intput and output in an asynchronous manner. For doing this, Asio as multiple contexts and constructs to deal with multiple threads and non blocking calls, and who executes them, and when they are executed.

The problem is: If I just want to connect to the network and exchange data, do I need to worry about io-context and handlers, and executors? Probably not.

Keep It Simple, Stupid.

In the C++ world, we struggle at keeping simple thing simple. The collection of libraries from the Boost project is one good example. Don’t get me wrong, these are high-quality, peer-reviewed C++ libraries. They are good code written by smart people, with the seal of approval by other smart people.

They are a demonstration of what you can do when you want to push the language forward. They contains a lot of useful pieces of generic code that you can reuse. A lot of really important and useful things from Boost finally landed inside the C++ standard library since 2011, like smart pointers, chrono, array, lambdas, and probably a lot more things like that are going to jump from existing on boost, to being implemented inside the standard.

And, if today you ask for a recommendation of something to use to do networking code in C++, I’m almost sure that you’ll get pointed to either Asio, the version of Asio inside of Boost, or, the Networking TS (addition to the standard that will probably land in a future version of the C++ language) that is… Basically based on Asio.

As you can guess… It’s not like I don’t like Asio, I find it genuinely interesting, and potentially useful. But I’m unsure it is the thing that I want to see standardized.

As stated earlier, if you’re just going to do some TCP/UDP communication, Asio comes wrapped in unnecessarily complexity.

Moreover, your OS comes with a socket API, but it isn’t super convenient to use in C++, and it’s not portable without doing the ugly #ifdef preprocessor dance.

Few months ago, I was thinking about this situation, and thought why not just wrap the os library in a nice C++17 interface?

Introducing kissnet

Kissnet is a little personal project I started during the summer, and that I tweak a little from time to time, it’s mostly for fun, but I feel like some people could have some use for something like this.

The design goals of kissnet are pretty straightforward:

  • Be a single header library, nothing to link against
  • Be a single API for all supported operating systems
  • Use C++17 std::byte as a “safe” way of representing non-arthmetic binary data
  • Be the lighthest/thinest possible layer on top of the operating system as possible
  • Handle all (or most) of what TCP and UDP sockets can do in ipv4/v6
  • Don’t require the user to worry about the endianess of the network layer.
  • Just transport the bytes, and do nothing else
  • Hide os-specific weirdness
  • Optional exception support (You can chose to replace throwing exception with the program either aborting or calling your own error handler)
  • stay simple

Kissnet only implement 3 kinds of objects:

  • A socket class. The behavior of the socked is templated around the protocol used (TCP vs UDP) and the version of IP used (ipv4 vs ipv6)
  • An endpoint class that permit you to specify a host and port as just a string and a number, or a “hostname:port” string.
  • A “buffer<size>” class that is just syntactic sugar around an std::array<std::byte, size>

Buffer are for holding received data, buffer know their own size, and can read the correct amount of bytes for you. Kissnet doesn’t care what data is sent, it’s not kissnet job.

Sockets are non-copyable (but movable) objects and they have the typical operations you can apply on socket implemented (bind, listen, accept, connect, send, receive).

Kissnet automatically manages the initialization/de initialization of the underlying socket API if needed (like on windows). This is done by exploiting the reference-counting of std::shared_ptr<>. This is the only overhead on top of holding a socket “file descriptor” ( = an simple integer variable).

I’ve only used kissnet in a couple of toy programs and not in a real project, however, I already think that I prefer this simple, down to the metal but yet type-safe and cross platform library as using something like Asio. Asio feels like using a bazooka to do kill a fly. I’ve heard a few opinions going the same way as I do. This is why I’ve put this little experimental project on GitHub, under a permissive MIT license.