A review of D2 – constness and immutablility

The D programming language is a very promising language. I propose here to do a quick review of some problems it is facing (or at least problems I’m facing when using it) and propose some solutions.

Immutability and constness in D2

I wanted to to a single, but writing it, it kept getting longer and longer. The point wasn’t to rant randomly, but also to make some proposal to improve things. So I did split it into smaller articles, and here the first one. We will discuss constness and immutability in D2.

Immutability

D2 has several types qualifier (immutable, shared) and they are transitive. It means that any reference or pointer cannot point to anything depending on its type qualifier and cast cannot be done in all directions.

immutable data are data that can only be set once and never redefined again. This ensure that immutable data can be shared without any synchronization process. By default, data are thread local storage, and immutable is a way to share data across threads (with some limitations).

immutable data can point to immutable data only.
mutable data can point to mutable or immutable data.

This ensure that immutable data can never point to thread local data, so it avoid sharing data that isn’t supposed to by mistake.

Conciliate mutable and immutable world : const

Theses qualifiers are completed by const. const is a wild-card, because both immutable and mutable data can cast implicitly to it. It allow the programmer to write code that will work with both mutable and immutable data without duplication when modifying data isn’t required.

So you can cast immutable and mutable to const, but not the other way around.
mutable data can point to const but not the other way around.
const data can point to const or immutable data.
immutable data cannot point to const.

Theses rules ensure that previously expressed rules still apply.

Consider the simple exemple :

1
2
3
4
5
6
7
8
9
10
11
const(int)* fun(const int* p) {
    return p;
}

void main() {
    int a = 3;
    immutable int b = 5;

    fun(&a);    // Works
    fun(&b);    // Works too !
}

Great, now we have a function that will work both with mutable and immutable.

Going further with inout, a scope defined type qualifier

Sometime, returning const just doesn’t make it. Because if you pass an immutable as argument, you may want to return an immutable, and not a const. This is explained in this article about inout very nicely.

Consider the simple example from previous part of the article :

1
2
3
4
5
6
7
8
9
10
11
12
const(int)* fun(const int* p) {
    return p;
}

void main() {
    int a = 3;
    immutable int b = 5;

    const int* c     = fun(&a);    // Works
    int* d           = fun(&a);    // Fails ! However, with cut paste of the function code, it works.
    immutable int* e = fun(&b);    // Fails ! But will also be correct with cut paste programming skills.
}

If you call fun with a immutable parameter, you’ll loose this qualifier for a weaker one : const. A solution is to duplicate the code for immutable and mutable, or to make the compiler duplicate the code using metaprogramming. But it is not required here. We need a way to express the return type according to the parameter type.

Then come inout. inout is a local type qualifier that can refers to mutable, immutable or const data depending on the case. Locally, inout data are considered as const, but outside this scope, data will get its original type qualifier back. It goes like :

1
2
3
4
5
6
7
8
9
10
11
12
inout(int)* fun(inout int* p) {
    return p;
}

void main() {
    int a = 3;
    immutable int b = 5;

    const int* c     = fun(&a);    // Works
    int* d           = fun(&a);    // Works
    immutable int* e = fun(&b);    // Works
}

The addition of this functionality was a great step forward to reduce the type qualifier mess. But some problems remains.

Existing code that has been written without const or immutable in mind.

Phobos, the standard lib of D, is full of code that isn’t const compliant. To begin with, the toString member function of Objects isn’t const, so cannot be called on const/immutable objects. This is problematic for such a feature, that is required everywhere.

Another example is invariant. Both feature are not supposed to modify the object they ran on (it would even be a huge mistake in invariant).

The sames goes for toHash, opCmp, opEquals, and so many others all over the place !

The current situation is very problematic. In the current state of things, const is broken. It is one a most important problem a beginner encounter and scared beginner isn’t what we want to make that language successful.

This will require breaking API changes. We have no choice, we’d better admit it and plan the change. No easy solutions here, just painful ones. The sooner we do that, the smaller the codebase will, the smaller the impact will. Let’s not be afraid and break our code !

A widely spread misconception in the D community : const cannot qualify a function

In this part, I’ll use const everywhere. However, the whole thing apply to immutable as well. I just omit it to avoid repeating everywhere that the same goes for immutable.

Let’s look at the code below :

1
2
3
const int* fun(const int* p) {
    return p;
}

Know what it does ? It does not compile ! « Error: function fun.fun without ‘this’ cannot be const/immutable » is saying the compiler. Thank you dude !

It trigger an error because the compiler try to qualify not the return type, but the function. What is a const function ? In fact it make sense in a very specific case : non static class member functions. In all other case, expect if you put parenthesis, as in previous code samples, it will not compile. Want to scare beginners ? Here is a good solution !

Note that this specification is even against D design principle « if it look like C, it should behave as expected by a C developer ».

Ok, that being said, and considering it is just annoying for most function, let’s consider the case of non static class member function :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class Foo {
    int* x;
   
    // Error: cannot implicitly convert expression (this.x) of type const(int*) to int*
    // Excaclty the reverse of what you would expect of such a code at a first look.
    const int* getX() {
        return x;
    }
   
    // Error: cast(int)*this.x is not an lvalue
    // Yes ! ref is qualifying the return type (int), but const is qualifying the hidden parameter this. How convenient and intuitive !
    ref const int getX2() {
        return *x;
    }
   
    // Hopefully this is working as expected. The postfix const syntax is what what is often preferred in current D code, because the other way is too confusing. The parenthesis ensure that the first const is qualifying the return type.
    const(int)* getX3() const {
        return x;
    }
}

I think we could simply admit that const isn’t qualifying the function, but the hidden parameter this. This has no advantages whatsoever and just confuse newcomers. Additionally, this break readers automatisms, because reading const int* doesn’t mean always the same thing.

The worse case being the « ref const » joke in the return type.

D aims to break patterns of error, and by mistake introduced one that didn’t existed in its predecessors. It has to be fixed.

This pattern can be encountered in 3 cases :

  1. Standard functions, or static member functions.
  2. Non static member functions.
  3. Function pointers and delegates types.

And we want to qualify different things as being const :

  1. The return type.
  2. The hidden parameter – this or context execution of a delegate.
  3. In case of function pointers and delegates types, the pointed memory location.

Let consider the most simple example : a function. The only things that make sens in this case is const qualifying the return type. So why overcomplicating ourselves ? Let’s make const in the return type qualify the return type itself instead of outputing error. It is what most people will expect.

1
2
3
const int* fun(const int* p) {
    return p;
}

Then, comes hidden parameter. I strongly suggest postfix notation for that, so it cannot be confused with return type qualifier, and would break the confusion about const qualifying the function. This is already know as a good practice in most D project, and shouldn’t break too many code. However, it should be possible to put that const anywhere after the return type.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class Foo {
    int* x;
   
    const int* fun() {  // OK
        return x;
    }
   
    const int* const fun() { // OK
        return x;
    }
   
    int* const fun() { // Fail : const int* cannot be casted as int*
        return x;
    }
   
    const int* fun() const { // OK. Equivalent to proposal 2.
        return p;
    }
}

Then we need to be able to qualify function pointer and delegate. Let’s start with function pointer, because it doesn’t have transitivity issue. In such a case, the same as previously exposed apply : const qualify return type if it part of it, and qualify function pointer after that. Note that member function are delegate (they have hidden parameter) so this will not conflict. For delegate, transitivity impose us that the type qualifier of the delegate qualify also the hidden parameter. In such a case, const after return type will qualify both delegate and hidden parameter, and postfix const will qualify only the hidden parameter.

Let’s put that into some code so it’s easier to understand :

1
2
3
4
5
6
7
8
9
10
11
12
13
int* function() fp; // A function pointer that return an int*.
const int* function() fp; // A function pointer that return a const int*.
int* const function() fp; // A constant function pointer that return an int*.
const int* const function() fp; // A constant function pointer that return a const int*.

int* delegate() dg; // A delegate that return an int* using a mutable hidden parameter.
const int* delegate() dg; // A delegate that return a const int* using a mutable hidden parameter.
int* const delegate() dg; // A constant delegate that return an int* using a const hidden parameter (because of transitivity).
const int* const delegate() dg; // A constant delegate that return a const int* using a const hidden parameter (because of transitivity).
int* delegate() const dg; // A delegate that return an int* using a const hidden parameter.
const int* delegate() const dg; // A delegate that return a const int* using a const hidden parameter.
int* const delegate() immutable dg; // A constant delegate that return an int* using an immutable hidden parameter.
const int* const delegate() immutable dg; // A constant delegate that return a const int* using an immutable hidden parameter.

In last cases, I used immutable because const was implied by transitivity. So something stronger was required.

In all cases, parenthesis can be used to explicitly use const as a prefix. This would also make this modification auto compliant. Basically, auto should behave as if it as parenthesis around it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
const(int* function()) fp; // A constant function pointer that return an int*.

const auto fp = function int*() { // A constant function pointer that return an int* as well.
    return null;
}

const auto fp = delegate int*() { // A constant delegate that return an int* as well using a const hidden parameter.
    return null;
}

auto fp = delegate int*() const { // A constant delegate to a function that return an int* using a const hidden parameter.
    return null;
}

auto const fp = delegate int*() const { // A constant delegate to a function that return an int* using a const hidden parameter.
    return null;
}

auto fp = delegate int*() immutable { // A delegate to a function that return an int* using a immutable hidden parameter.
    return null;
}

const auto fp = delegate int*() immutable { // A constant delegate to a function that return an int* using a immutable hidden parameter.
    return null;
}

Conclusion

I hope I didn’t scarred you. D2 type system is really nice and address many issue that exists in other languages, introducing the tools required to handle functional and concurrent programming style.

However, this type system is really something new and some work has to be done to make it as powerful and as easy to use as it is on the paper.

Leave a Reply

Your email address will not be published. Required fields are marked *