Language:

en_US

switch to room list switch to menu My folders
πŸ‘ˆ #0 - intmax_t and Poking at ABI

πŸ‘‹

It's Letter #0, and we're starting it off with a new feature idea!

If you missed one of the related articles written by our very own "ThePhD", catch it here. Essentially, the gist of it is that C doesn't have the built-in language tools to solve some of its most pressing existential problems. This doesn't mean C (and, by extension, C++) can't be used to make great APIs, it just means that C's ability to evolve into something better and handle greater use cases it a little questionable!

For that reason, we're thinking about actually introducing a new feature to C. We might even get to prototype it out in GCC or Clang! But, the idea is very simple. If you're used to C++'s "type aliases" or C's "typedefs", then the concept here is pretty straightforward for a new feature. Here's the core idea, using made up syntax:

int a_function (int a, int b) {
    return a + b;
}

using a_function_by_any_other_name = a_function;
// does it smell as sweet?

Essentially, it allows you to provide a "transparent" alias for a pre-existing function declaration. It allows you to interact with that transparent alias like a normal function call:

int main () {
     int value =
          a_function_by_any_other_name(3, 1);
     return value; // returns 4
}

You can even do normal things with it, like take its address or let it convert to a function pointer to pass it as an argument to another function. Note that you can get similar behavior (but not perfect) in C++ using constexpr an references:

int a_function(int a, int b) {
    return a + b;
}

constexpr const auto& by_any_other_name( = a_function;

int main() {
    return by_any_other_name(1 ,2);
}

This means it's not totally necessary in C++. But for C β€” which lacks both constexpr and references β€” it is perhaps the only way to have a non-macro solution to "renaming" functions.

The reason being able to provide transparent aliases is you have to avoid things like Shared Libraries from containing the exact symbol by_any_other_name! This is important for ABI, for example if you wanted to direct a function call to a specific internal implementation but give the user a "standard" name to call. For example, one can map the cos global function to a implementation-specific intrinsic:

using cos = __cosf64;

This is mostly for people in the weeds of implementing heavily-depended-upon libraries, but it also comes in handy for other folk who want to make certain functions available without having to redeclare/rewrite them under a different name and use implementation-defined tricks to have them show up.

A more full specification-like sketch can be found here in this Godbolt, showing the behavior we would want overall. (It doesn't compile, because it's a new idea.) Maybe, one day, it will be proposed for standardization in C!

β€” Shepherd's Oasis πŸ’™



https://buttondown.email/Soasis/archive/0-intmax_t-and-poking-at-abi/

Posted by rss <> on Sat Dec 12 2020 08:49:38 UTC
0 comments 0 new | permalink
🎈 #1 - The Conference and the Priorities

πŸŽ‰!

Happy Holidays and wonderful cheer to each and every one of you! It's Letter #1, and there's a lot that's been going on!

A New Conference

As was announced on Twitter, there's a new conference in town:

the CoSy Tech Con!

It stands for the (Co)mputing (Sy)stems (Tech)nology (Con)ference, but honestly that's an enormous mouthful. So just CoSy (pronounced like cozy) is good!

Why CoSy?

The motivation for the conference was two-fold. The first reason is quite simple: we needed a place where people beyond C++ developers could talk about the systems they work on and develop. We decided that we would make a conference that was going to embrace not just C++ developers, but C, Rust, Zig, FORTH, Nim, Fortran, Go and many, many more languages.

Part of being inclusive and creating a space where multiple people can come together is to do exactly that: a conference where languages become tools rather than the dominating feature of a given talk or -- indeed -- a conference!

The second reason is because it became very hard to keep justifying some of the abuse of the people providing significant sponsorship and donations to other C++ conferences. From the N-word to actually screenshotting folk's words out-of-context and sending it to their jobs to be fired, there's a lot the leadership of previous spaces have done to make tech hostile.

Asking the people in charge to change has not worked now, or the last 20+ times we've asked. So, well, it's time to vote with our wallets and our feet, and take the leap of faith for a better world! We've got a strong focus on accountability and transparency with how we're going to be running this conference and protecting the most vulnerable, which is why we're going to be sharing some of our monetary priorities below!

List of Priorities

There are a lot of folk who have offered their help, so to help keep both us and them focused, here is a preliminary list of the things we're focusing on funding to deliver the conference to you:

  1. Platform Fees (just for hopin.to for interacting with others);
  2. Live Transcription (English) of all talks for all days to help the hard of hearing;
  3. Post-conference subtitling on all recorded and released videos (if Speaker consents to release);
  4. Accumulating additional funds to pay every single speaker; and,
  5. Finally, pay ourselves.

Because this is a free conference, we have about 10 months to cover the first three. The last 2 will be cherries on top for the free event. We plan to essentially spend every free dollar either improving the event by getting more skilled professionals to help us stitch the virtual event together, or paying out more and more to the speakers. We already have some ideas for generating revenue that we can put towards putting money in the Community's pocket, and we'll talk about these initiatives soon!

If you would like to get ahead of the curve on asking to be a speaker (for a Keynote or a regular talk), want to discuss early sponsorship, or have other things you'd like to let us know about, feel free to contact us here.

Keep an eye out for the Newsletter closer to the end of February. Submissions will be opening up a little after that, but folks like you will get the early access!

Thanks for supporting the dream. See you in the next 2 letters, next month!

β€” Shepherd's Oasis πŸ’™

PSJ: Lately, one of us have been reading a book that has been impossible to put down. Which, as it turns out, makes a lot of sense, since it's about anti-gravity!



https://buttondown.email/Soasis/archive/1-the-conference-and-the-priorities/

Posted by rss <> on Sat Dec 26 2020 11:00:00 UTC
0 comments 0 new | permalink
πŸ’£ #2 - Manually Destroying a Map For Space, and Fun!

πŸ’£ 😈

Happy middle of the month, everyone! If you are reading this, this meant that you have survived this far, and we're heckin' proud of you! This half-month's newsletter is going to be dedicated to std::unordered_map: or, specifically, how we saved some space on some critical devices by getting creative with how we share things!

But first..

CoSy!

A huge, big W E L C O M E to everyone who has signed up about CoSy! As with the last newsletter, here's an update of where we are:

  1. We are reaching out for sponsors and connecting with a lot of organizations! If you'd like to sponsor us, send an e-mail to cosy@soasis.org! We have a little two-page primer you can have, and can talk to your organization more directly if you like!
  2. We have contacted 6 Live Caption companies, who are getting back to us with quotes so we can make sure we support Hard of Hearing folks.
  3. We have a pretty good idea of what platforms we will stream on. More on that in the next Newsletter!
  4. We are offering trainings at the Conference! We have had immense interest in folks signing up to teach. If you're interested please send an e-mail to inquiries@soasis.org!
  5. We are working on STICKERS! (And a cute mascot, too!) Stay tuned for more on that.

We're excited at the progress we're making! So far, everything looks to be shaping up nicely. And now, let's get ready to...

πŸ’₯ Explode an unordered_map, manually!

The coding portion of this is about a technique a few people helped us put together. Essentially, the problem is there was code that needed to loop through a map and invoke a function that changed both the key and the value of a std::unordered_map, which is C++'s standard associative container (typically called a "dictionary" in Python and C#). The code looked like this:

int super_cool_computation(
    important_t* necessary,
    const good_t& params)
{
    using key_t = manual_destroy_key;
    using value_t = manual_destroy_key;
    std::unordered_map<key_t, value_t> my_map;

    // ...
    // used here to do some super cool stuff
    // ...

    for (auto& key_value_pair : my_map) {
        key_t& key = key_value_pair.first;  
        value_t& value = key_value_pair.second;
        key.clean_up_resource(necessary, params);
        value.clean_up_resource(necessary, params);
    }

    // my_map destroys normally after return
    return 0;
}

Unfortunately, there's 2 problems with this!

It Doesn't Compile!

This line:

key_t& key = key_value_pair.first;

Is illegal. key_value_pair.first is a const-qualified object! So, we can't get a normal reference type manual_destroy_key&, we need to get a constant reference type const manual_destroy_key&. Unfortunately, clean_up_resource(...) modifies things, so we can't call it if we const-qualify key!

We could use const_cast to get rid of the const-ness of the key_value_pair.first:

    // ...
    for (auto& key_value_pair : my_map) {
        key_t& key =
          const_cast<key_t&>(key_value_pair.first);
        // ...
    }
    // ...

This works, but...

It's effectively a lie!

The key in map types in C++ are created and stored as const. Casting it away means that, while unlikely, some day the optimizer could do things to that value not expecting us to mutate it. This eventually opens us up to Undefined Behavior (😱)!

Making clean_up_resource into a const method and then marking some members mutable might work, but it's essentially just moving the problem inside the clean_up_resource call.

It turns out, there's some C++17 methods for getting both the key and value out of a map in their non-const forms. It's called...

extract(iterator)!

If our goal is to iterate over a map and call a function to mutate the keys and values, extract is the go-to way to do it. So, using that, we can achieve a standards-compliant way of cleaning out a map and cleaning up BOTH the keys and values like so:

int super_cool_computation(
    important_t* necessary,
    const good_t& params)
{
    using key_t = manual_destroy_key;
    using value_t = manual_destroy_key;
    std::unordered_map<key_t, value_t> my_map;

    // ...
    // used here to do some super cool stuff
    // ...

    auto first_iterator = my_map.cbegin();
    auto last_iterator = my_map.cend();
    while (first_iterator != last_iterator) {
        // save an iterator to what we want to extract
        auto saved_iterator = first_iterator;
        // move the iterator, once every loop!
        ++first_iterator;
        // extract it!
        auto extracted_node =
          my_map.extracted(saved_iterator);
        key_t& key = extracted_node.key();  
        value_t& value = extracted_node.mapped();
        key.clean_up_resource(necessary, params);
        value.clean_up_resource(necessary, params);
        // let the node fall off!
    }

    // my_map is all empty!
    return 0;
}

The extract method returns a special node type, which has both the key and value. When we let the node type expire by reaching the end of the for loop's scope, it deletes the type and nothing gets inserted back into the map. This means you can effectively extract, but never replace!

Want to keep it instead of getting rid of it?

If you want to modify the key and also return it back to the origin, you can always call my_map.insert(std::move(extracted_node));. We don't do that here because we just wanted to clean out my_map!

This is useful when you can't store the necessary parameters inside the key and value classes. For example, a process was inserting hundreds of thousands of keys into one of the maps previously, and we had to duplicate necessary, params as members on every single key and value! This meant we were shedding tons of bytes - plus padding - and needing to allocate way more space than was necessary. Since the necessary parameters were identical for all keys and values, it made sense to pull them out and then manually clean up the type instead.

And, that's it! Congratulations, you made it to the end of one of the longer newsletters! The ones in the future will be shorter, and the next one may even contain a bit of a musical surprise, from us! 🎢

β€” Shepherd's Oasis πŸ’™



https://buttondown.email/Soasis/archive/2-manually-destroying-a-map-for-space-and-fun/

Posted by rss <> on Sat Jan 16 2021 00:55:56 UTC
0 comments 0 new | permalink
πŸ€“ #3 - Nerding Out

πŸ€“

Happy end of the month, everyone! Today, we’re gonna talk about fun things that can help you write some pretty snazzy C code, when you’re tired of formatting strings and other weird stuff. It will rely a tiny bit of vendor extensions (that might not be vendor extensions, soon!) so you can see the magic.

But before we get there…

⭐ CoSy Tech Con!

Most of you signed up to hear about CoSy! Progress is, of course, still being made:

  1. We are still reaching out for sponsors and connecting with a lot of organizations! If you’d like to sponsor us, send an e-mail to cosy@soasis.org! We have a little two-page primer you can have, and can talk to your organization more directly about benefits!
  2. We have a budget for how much Live Captioning should cost us, so we now know almost everything
  3. The official page that will also contain hallway tracks and other fun events will be at Hopin!
  4. We are going to be locking in training slots for people who have shown interest in training! If you’re interested still interested please send an e-mail to inquiries@soasis.org!
  5. The stickers and other cool things are now being worked on by artists!

All in all, things are moving ahead steadily. We hope that by March and April, we can start announcing the various trainings that will be happening and show off the new submission system!

One final note is that we’ll also possibly be increasing the conference to be 3 days, but taking less time each day. This might prevent people from needing to sit at their computers for too many hours (even if we give them breaks), and make the content accessible to a larger sum of folks since it doesn’t span the whole working day. And with that talked about, we are now officially going to…

COMPLETELY Nerd Out, with C

We’ve all seen it, before:

printf("%s", some_string);

printf. Friend of the debugger-less, revealing muse of ordering problems, champion of the 12 AM beleaguered student. Also, way too verbose and way too dangerous. Consider, for example, what happens when the equivalent printf("%d", some_string); happens, or even printf("%s", some_integer);. On some compilers, you get warnings when you do this! For example, with GCC:

error: format '%d' expects argument of
type 'int', but argument 2 has
type 'char *' [-Werror=format=]
     printf("%d", some_string);
            ^----------------

Most compilers implement checks for the format string, to make sure you’re matching the type of what you specified. But, it only works on string constants that the compiler is capable of reading! This works, but it’s… kind of frustrating! Wouldn’t it be better, if we had a way to know the type up front and just act on that? C doesn’t have templates like C++, but it turns out with a little magic, we can get somewhat close to it in C…

Can we divine the type of an expression given as an argument, and create variadic function calls that way?

πŸ” C features, A Closer Look

As said earlier, C doesn’t have templates. There’s no compile-time variadics, no type traits, nothing but our bare hands and some macros. So, we have to write this…

enum arg_type {
    arg_type_int,
    arg_type_double,
    arg_type_ptr_char,
    // ... and more...
    arg_type_shrug
};

This is our way of knowing what type we got. We’ll use it in a hot second, here. Next, we need something that will wrap each argument. Again, we don’t have templates here, so we’re going to use the one and only true type erasure we need: void*:

typedef struct arg_ {
    enum arg_type type;
    void* data;
} arg;

Now, upon seeing a pointer, many programmers would reflexively assume that we’re going to use some kind of allocation or other shenanigans to produce a pointer. But we don’t have to, thanks to this sweet feature…

Compound Literals

Compound Literals is a feature that does not exist in C++, but does in ISO Standard C. This is one of the ways where C code is not only different from C++, but far more powerful. Compound literals allow you make an

“unnamed object initialized by the initializer list. If the compound literal occurs outside the body of a function, the object has static storage duration;otherwise, it has automatic storage duration associated with the enclosing block.”
β€” Β§6.5.2.5 Compound Literals, Semantics, Paragraph 5

What this means is that if you write this:

void foo () {
    {
        int compound_literal_arr* = (int[]){1, 2, 3, 4, 5};
        /* Lots of work */
        printf("%d", *(compound_literal_arr + 2)); // prints 3
        /* Lots more work */
        // compound_literal_arr is still alive here!
    }
    // compound_literal_arr is gone here :(
}

You get an object that lasts for the duration of compound_literal_arr‘s scope (“automatic storage duration associated with the enclosing block”). This is incredibly useful, because it means we can hoist arguments into a higher lifetime that allows us to take pointers to them! In particular…

A function that takes anything

We can use what we just learned to create arguments whose lifetimes we know are long enough for us to do things like print them out. So…

void process_arg(arg arg0) {
    switch (arg0.type) {
        case arg_type_int:
            {
                // points at a single integer
                int* value = *(int*)arg0.data;
            }
            break;
        case arg_type_double:
            {
                // points at a single double
                double* value = *(double*)arg0.data;
            }
            break;

        case arg_type_char:
            {
                // points at a character string
                char* value = *(char**)arg0.data;
            }
            break;
        /* and so on, and so forth... */
        default:
            break;
        }
}

int main () {
    // create a "arg" struct, pass it into the
    // function with the right type, and with the value we want
    int value = 1;
    process_arg((arg){ arg_type_int, (int[]){ value } });
    return 0;
}

This obviously works, but:

  • We are still spelling out the type
  • We have to wrap it in that ugly-looking Compound Literal syntax

Can we abstract away some of this? Well, we can! Enter…

Enter _Generic

_Generic is a tool most C programmers shouldn’t be using. But, occasionally, it can come in handy. It’s an expression type, and it takes a list of TYPE : EXPRESSION associations:

void process_arg(arg arg0);

int main () {
    int value = 1;
    arg_type type = _Generic(value,
        int: arg_type_int,
        double: arg_type_double,
        char*: arg_type_ptr_char,
        default: arg_type_shrug
    );

    process_arg((arg){ type, (int[]){ value } });

    return 0;
}

We can move this into a macro, for clarity:

void process_arg(arg arg0);

#define GET_ARG_TYPE(value) _Generic(value, \
    int: arg_type_int, \
    double: arg_type_double, \
    char*: arg_type_ptr_char, \
    /* and so on, and so forth... */ \
    default: arg_type_shrug \
)

int main () {
    int value = 1;
    process_arg((arg){ GET_ARG_TYPE(value), (int[]){ value } });

    return 0;
}

Nice! We still have one problem: (int[]) is still an explicit reference to the fact that we’re trying to create a compound array literal of type int. How do we make it rely solely on “value”? Well, we could try creating a _Generic that produces a void*, but the problem with _Generic is that it’s an expression. Every branch of _Generic gets evaluated, which means if you try to make a _Generic tree of this, you will get some weird errors for different types:

#define GET_ARG_DATA(value) _Generic(value, \
    int: (void*)((int[]){ (value) }), \
    double: (void*)((double[]){ (value) }), \
    char*: (void*)((char*[]){ (value) }), \ // !!!
    default: (void*)NULL \
)

int main () {
    int value = 1;
    // ERRORS:
    GET_ARG_DATA(value);
    // initialization of 'char *' from 'int' makes 
    // pointer from integer 
    // without a cast [-Werror=int-conversion]

    return 0;
}

πŸ˜“ Ouch.

_Generic does not have “template-like” properties, even in a Macro: all branches must be valid and compile for all types its exposed to. This makes it much less good for us… until we realize we skip the _Generic altogether, with a little extra help from a widely-implemented extension called __typeof:

#define GET_ARG_DATA(value) _Generic(value, \
    int: (void*)((int[]){ (value) }), \
    double: (void*)((double[]){ (value) }), \
    char*: (void*)((char*[]){ (value) }), \ // !!!
    default: (void*)NULL \
)

#define GET_ARG_DATA(value) \ 
    ((__typeof((value))[]){ (value) })

void process_arg(arg arg0);

int main () {
    int value0 = 1;
    double value1 = 2.0;
    const char* value3 = "3";
    process_arg((arg){ GET_ARG_TYPE(value0), GET_ARG_DATA(value0) });
    process_arg((arg){ GET_ARG_TYPE(value1), GET_ARG_DATA(value1) });
    process_arg((arg){ GET_ARG_TYPE(value2), GET_ARG_DATA(value2) });
    return 0;
}

And, we add some more macros, just to make it less spammy:

#define GET_ARG_TYPE(value) _Generic(value, \
    int: arg_type_int, \
    double: arg_type_double, \
    char*: arg_type_ptr_char, \
    /* and so on, and so forth... */ \
    default: arg_type_shrug \
)



#define GET_ARG_DATA(value) \
    ((__typeof((value))[]){ (value) })

#define GET_ARG(value) \
    (arg){ GET_ARG_TYPE(value), GET_ARG_DATA(value) }

void process_arg(arg arg0);

int main () {
    int value0 = 1;
    double value1 = 2.0;
    const char* value3 = "3";
    process_arg( GET_ARG(value0) );
    process_arg( GET_ARG(value1) );
    process_arg( GET_ARG(value2) );
    return 0;
}

Very good! There is even more you can do to make this work out.... but we’re going to leave that…

For next time. 😁

It’s a lot to put in one session, so we’re going to break it up a little bit!

We’ll pick this up next Newsletter, and talk about the preprocessor’s __VA_ARGS__, variable arguments to functions using ..., and how this concept can be extended to have a format like…

int main () {
    process_args(1, 2.0, "3");
    process_args(0x4, '5');
    process_args((char)0x36);
    return 0;
}

in C!

Until then, have a look at this C++ paper proposing Compound Literals to the Language. There’s also a paper to make typeof work in C, and not have it be a “widely implemented extension” anymore.

See you next time!

β€” Shepherd’s Oasis πŸ’™

P.S. With end of the first month of the New Year, we here at Shepherd’s Oasis decided we’d share our New Year’s Resolution! Right now, it’s just 1600 x 900, but maybe we’ll update to 4096 x 2160 in this New Year.



https://buttondown.email/Soasis/archive/3-nerding-out/

Posted by rss <> on Sun Jan 31 2021 22:54:28 UTC
0 comments 0 new | permalink
✍️ #4 - A Great Step Forward for C++

✍

Happy start of the month! This is going to be an incredibly short Newsletter that is going to cover what we’ve been up to for all of February, and why we haven’t sent out some Newsletters during the month of love!

CoSy ⭐ Tech Con!

The date is now fully settled! And all of you reading this Newsletter will be the first to know:

  • October 18-20th, 2021: Main Conference
  • October 21st, 2021: Teaching & Trainings

Save the dates! Submissions will be opening in a few weeks as well. We are excited to have people submit their talks. As a reminder, we are going to be employing a double-blind system, where both submitters and reviewers will not know who is looking over their work. We are confident this will help reduce any bias in the system from names, country of origin, ethnicity, perceived gender, or otherwise! We are also going to be speaking over the coming months at various different universities, podcasts, and more to make sure we get the word out to community members not traditionally encircled with Systems Programming. (If you have a place – a college/university, a school, a video, a podcast, anything β€” we will be more than happy to talk to you about what we’re doing here!)

As usual, we’re still accepting any new sponsors that would like to come along. We’ve had a very positive reception and we’re glad to make sure you, your company, and/or your organization are given the recognition and perks they deserve during the conference! Feel free to reach out to us at cosy@soasis.org.

Trainings are being solidified! We’ve got a few folks who are ready, and we are reaching out to anyone else! We are interested in both established trainers and new-style trainers, just so long as you’re ready and willing to teach and offer a great one-day workshop or lecture. We will aggressively market you in any promotional material that happens, and help get as many people signed up for your courses as possible, so if you’re interested please reach out!

And now, for the thing that’s kept us busy when CoSy was not…

Better Text in C++

We are extremely excited to announce that after working very hard with JeanHeyd, we’ve created the first and most integral part of a library that works with text in C++.

ztd.text

This has been a long time coming. JeanHeyd has spoken of this idea for a year publicly and has spent much more time privately curating and preparing the source. It’s finally released and open source: https://github.com/soasis/text.

One of the reasons this took us a lot longer to get out the door is because this time around, we wanted to document everything first. Not just the API, but the Design and the Philosophy behind that design. We also got to include nice documentation images that come pretty close to being what the kids call “shitposts” these days:

We like it quite a bit!

There are plans to write some tutorial blog posts on how to use the library to define your own encodings and get plugged into the ecosystem for very little effort. JeanHeyd’s design is pretty slick, and we have nothing but praise for the Lucky 7 encoding design. He’s built it off a solid foundation of prior art and new advancements, and bridges a few corner cases and design gaps other attempts at in this space had.

So, go explore the documentation and code, and let us know how you feel about it by e-mailing opensource@soasis.org! And, of course, if it looks useful to you, absolutely do pick it up and take off to the races with it!

Wait… What about that __VA_ARGS__ stuff from last time?!

Ah, you have a good memory! But do not fret: we’re going to go over that too! We said “next time”, but we wanted to highlight the accomplishments of the last month and the things we are delivering on. We’ll get to more fun things using C and macros later. πŸ€“

We also might have some spicy, C Standard-related news for the next half-month newsletter, too… πŸ‘€

See you next time!

β€” Shepherd’s Oasis πŸ’™



https://buttondown.email/Soasis/archive/4-a-great-step-forward-for-c/

Posted by rss <> on Tue Mar 02 2021 11:22:01 UTC
0 comments 0 new | permalink