Implementing toll-free bridging
Toll-Free Bridging is one of the really cool tricks in MacOS, allowing you to transparently use some Foundation (Objective-C) and CoreFoundation (plain C) objects in functions designed for their counterpart. In other words, you can pass an NSArray created in Objective-C to a CFArray function, or use a CFStringRef object as the receiver of an NSString message like [str length], and both will work. It's "toll free" because there is no conversion - the objects are the same structure.
Unfortunately, Apple's public sources of CoreFoundation and libobjc have this functionality removed. And we have our own Foundation that has no concept of it today. BUT .. we need this for ABI compatibility to compiled Mac apps. So how do we get there?
Aligning types
Every CF type has an ID number and a kNSCF*TypeID constant defined for it in Foundation's NSCFTypeID.h. These same type numbers are returned by [obj _cfTypeID] on ObjC objects regardless of actual bridging. First I aligned these between the two frameworks using CoreFoundation as the authoritative list. Next, the actual object layouts have to be aligned too, in libobjc, Foundation, and CoreFoundation.
Verifying object layouts
I had guessed most of this from context and experience, but as they say in the KGB: "trust, but verify." Using a tiny ObjC test on MacOS and the structure copied from CF Lite, we can see the structure of a bridged object:
#include <Foundation/NSString.h>
typedef struct {
uintptr_t _cfisa;
uint8_t _cfinfo[4];
uint32_t _rc;
} CFRuntimeBase;
int main(int argc, char **argv) {
NSMutableString *s = [[NSMutableString alloc] initWithCString:"some value"];
printf("s addr: %p\n", s);
uintptr_t *u = (uintptr_t *)s;
for(int i = 0; i < 4; ++i) {
printf("s[%d] = %lx\n", i, u[i]);
}
CFRuntimeBase *base = (CFRuntimeBase *)s;
printf("cfisa %p cfinfo %lx rc %lx\n", base->_cfisa, *(uint32_t *)(base->_cfinfo), base->_rc);
return 0;
}
I use a mutable object here to make sure it doesn't create a constant string class. In the output below, s[0] is our isa class pointer, s[1] is a combined cfinfo and rc (32 bits each), s[2] is a pointer to the string's bytes, and s[3] is the string's length. So we know that bridged objects do contain the CF fields. We also know that cfinfo is populated with the CFTypeID (shifted left 8) and the ref count is initialized for both NS and CF objects.
% ./foo
s addr: 0x....
s[0] = 1d8001070d0519
s[1] = 1000007ad
s[2] = 7fc16560a1d0
s[3] = a
cfisa 0x1d8001070d0519 cfinfo 7ad rc 1
Using a regular NSString will result in a constant string class. s[0] is the isa, s[1] is cfinfo and rc, s[2] becomes the actual bytes of the string rather than a pointer, and the length follows the data. I can tell that s[2] is my string data because the bytes correspond to the ASCII values of "some value". This test confirms the layout of the NSCFString which is the same as a constant string (NSDarwinString in our Foundation):
@interface NSDarwinString : NSString {
char *_bytes;
unsigned _length;
}
@end
Our isa seems to be class __NSAtom. We don't have this class in Foundation so may need to create it for ABI compatibility. But for now we'll leave it as NSString.
Restoring the CoreFoundation macros
Apple's public CoreFoundation source (aka CF Lite) doesn't support toll-free bridging. It's not the source of CF on MacOS - just a subset pared down for other OSes. We need to restore the functionality that detects and dispatches to ObjC objects when bridging. This is driven by a handful of macros and two tables in CoreFoundation.
First, I had to figure out the likely implementation of these from the call sites:
-#define CF_IS_OBJC(typeID, obj) (0)
-#define __CFISAForTypeID(t) (0)
+#define CF_IS_OBJC(typeID, obj) (obj && (*(uintptr_t *)obj) \
+ && (typeID < __CFRuntimeClassTableSize) \
+ && (*(uintptr_t *)obj == __CFRuntimeObjCClassTable[typeID]))
+#define __CFISAForTypeID(t) (t < __CFRuntimeClassTableSize \
+ ? __CFRuntimeObjCClassTable[t] \
+ ? __CFRuntimeObjCClassTable[t] \
+ : (uintptr_t)_CFRuntimeGetClassWithTypeID(t) \
+ : 0)
CF_IS_OBJC tests whether a specific object is backed by ObjC or CF. A non-zero isa which matches the class registered in the ObjC runtime table for the object's typeID identifies ObjC. This is a Boolean value.
CFISAForTypeID returns the isa value for a CF type from the runtime type tables. If the type doesn't exist in the ObjC table, we check the CF table. These tables are 1024 entries each and have a 0 for any unused slot, consistent with the 0 returned by the previous macro. Therefore, they only trigger for types that have been registered as either CF or ObjC, which is done during initialization of the library. I'm still unclear why this function takes both a type and object.
I also had to implement these:
+extern id objc_msgSend(id, SEL, ...);
+
+#define CFTYPE_IS_OBJC(obj) (obj && *(uintptr_t *)obj != 0 \
+ && __CFRuntimeObjCClassTable[(*((uintptr_t *)obj+1) >> 8) & 0xff])
+
+#define CFTYPE_OBJC_FUNCDISPATCH0(rettype, obj, sel) \
+ if (CFTYPE_IS_OBJC(obj)) \
+ return (rettype)objc_msgSend(obj, #sel)
+
+#define CFTYPE_OBJC_FUNCDISPATCH1(rettype, obj, sel, a1) \
+ if (CFTYPE_IS_OBJC(obj)) \
+ return (rettype)objc_msgSend(obj, #sel, a1)
CFTYPE_IS_OBJC checks whether a specific object is backed by ObjC or CF.
A non-zero isa and a typeID (stored in _cfinfo shifted left 8) that exists in the ObjC runtime class table identifies an ObjC object. We may need to check that it matches the registered type as in CF_IS_OBJC.
A few allocation functions that set cfisa needed to be updated to set it to the return from __CFISAForTypeID(typeID) instead of just 0.
Wiring up NSObject
The key to toll-free bridging is that NSObject and CFRuntimeBase (which are the bases of all objects in both frameworks) have an identical first 16 bytes: isa (class pointer), class bits (aka cfinfo flags and refcount). I aligned the definition of NSObject in libobjc with CFRuntimeBase and added a hook in NSAllocateObject (Foundation) that sets the required CF type info on a new ObjC object:
@interface NSObject <NSObject> {
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wobjc-interface-ivars"
+ /* Keep aligned with CFRuntimeBase */
Class isa OBJC_ISA_AVAILABILITY;
+ uintptr_t bits;
+ struct objc_object *base = (struct objc_object *)result;
+ base->bits = (uintptr_t)[result cfinfo];
+ printf("NSAllocateObject: result %p class %s isa %p bits 0x%lx\n",
+ result, object_getClassName(result), base->isa, base->bits);
And of course the struct backing an object also needs the full isa_t definition to align.
struct objc_object {
Class _Nonnull isa OBJC_ISA_AVAILABILITY;
+ uintptr_t bits;
};
This is enough for the mechanism itself to work. The rest of the effort is in making the NS object layouts match the CF object layouts. (We assume that the CF layouts will match those of compiled Apple Foundation objects.)
Current state
So far, I have a good start on NSString and have introduced NSCFString. As seen in the (slightly edited for clarity) test script output below, static strings originating as either CFSTR("x") or @"x" are identical and can be used with either NSString selectors or with CFString* functions.
Native objects (objects created with their native method, either CFStringCreateWithCString or [NSString stringWithCString:]) are interoperable but not 100% correct yet. The object constructed by CFStringCreateWithCString is, ironically, making CFShowStr unhappy while the NSString works fine. Also, the storage I chose for NSCFString works, but does not support CFStringGetCStringPtr() so there is the null result below. I'll probably change this to the inline form for strings under 256 chars.
Bridged retain is working, transferring ownership out of ObjC.
Bridged release is not working yet. I suspect this is related to the CFShowStr problem.
So... getting there :)
root@ravynOS$ /root/testcfb
Setting up bridging
Registering ObjC class 0x10170b450 for type ID 7
--- Static strings ---
+ NSString(0x1015f9020): 0x101751e20 0x7c8 0x1015f8ddf
+ CFStringRef(0x1015f9040): 0x101751e20 0x7c8 0x1015f8df5
not inline
+ contents of s: testing a long string
+ contents of c: crud
not inline
+ testing objc cast: crud
+ testing c cast: testing a long string
Length 21
IsEightBit 1
HasLengthByte 0
HasNullByte 1
InlineContents 0
Allocator SystemDefault
Mutable 0
Contents 0x1015f8ddf
Length 4
IsEightBit 1
HasLengthByte 0
HasNullByte 1
InlineContents 0
Allocator SystemDefault
Mutable 0
Contents 0x1015f8df5
--- Native objects ---
+ NSCFString(0x10170b450): 0x10000078c 0x6163666f6f6c6615 0x61636e61796e2074 0x1
bits: flags 78c, rc 1
Length 25
IsEightBit 1
HasLengthByte 0
HasNullByte 0
InlineContents 0
Allocator 0x7fc2044414a0
Mutable 0
Contents 0x7fc204441520
This is an NSString, not CFString
+ length of s: 25
+ contents of s: hello this is a long text
+ length of c: 21
+ contents of c: floofcat nyancat fuff
+ testing objc cast: floofcat nyancat fuff
+ testing c cast: (null)
+ testing c cast buffer: hello this is a long text
--- test_CFBridgingRetain ---
Original ObjC object: 0x7fc204441770
CF object pointer: 0x7fc204441770
ObjC object pointer: 0x7fc204441770
Pointers match - toll-free bridging OK
Released CF object
--- end test_CFBridgingRetain ---
--- test_CFBridgingRelease ---
Created CF object: 0x7fc2044410b0
Bridged ObjC object: 0x7fc2044410b0
NS class 0x10170b720 (NSString)
+ nil(0x7fc2044410b0): 0x0 0x449e352286bd9cb 0x101baaec0
thread da on cpu0 - err=0000000000000004. loaded @00000001015f7000
CR2=0000000000000018 RSP=00007ffeee608d08 RBP=00007ffeee608d40 RFLAGS=00000246
RAX=000000000000003a RBX=0000000000000000 RCX=0000000000000002 RDX=000000010170b720
RDI=00007fc2044410b0 RSI=0000000101ba2bc3 R10=0000000000000000 R8=0000000101ba2bc3
R9=00007ffeee608cc0 R15=0000000000000000 R14=0000000000000000 R13=0000000000000000
R12=0000000000000000 R11=0000000101ba2bc3 RIP=0000000101b2b35d
CS:0000002b DS:00000000 ES:00000000 FS:00000000 GS:00000000
Stack:
00000001015f8af3
00000001021796f1
0000000000000001
Segmentation fault: 11
Update! Fri Aug 14
After fixing a few bugs in the integration, the NSCFString tests now fully pass :)
--- test_CFBridgingRetain ---
Original ObjC object: 0x7fdaf3406b00
CF object pointer: 0x7fdaf3406b00
ObjC object pointer: 0x7fdaf3406b00
Pointers match - toll-free bridging OK
Released CF object
--- end test_CFBridgingRetain ---
--- test_CFBridgingRelease ---
Created CF object: 0x7fdaf3406c90
Bridged ObjC object: 0x7fdaf3406c90
NS class 0x10cf68728 (NSString)
+ NSCFString(0x7fdaf3406c90): 0x10cf68458 0x60078c 0x66206f6c6c65480d
It is an NSString under ARC
--- end test_CFBridgingRelease ---