Archives

These are unedited transcripts and may contain errors.



IPv6 Working Group session
15th May 2013, 2 p.m.


CHAIR: Hello. It's two o'clock. Time for the IPv6 Working Group session, again. Here we are. A few people still queueing up at the coffee desk, but I'm sure they will join us, I am Marco Hogewoning; I work for the RIPE NCC. This Working Group has three co?chairs, Shane Kerr and David Kessens. If you have anything to talk about, suggestions for talks, suggestions for work to go to by our Working Group, we have an e?mail address, IPv6?WG?chairs [at] ripe [dot] net, that will end up in all three of our mailboxes.

What are we going to do today? We have got two sessions, more or less right now as you can see. Focuses a lot on implementing IPv6, how to write clean code, stuff you may encounter if you really start running IPv6 only networks, etc.

Session 2 is tomorrow, unfortunately 9 o'clock, so don't party too hard tomorrow. We do want you in here at 9 o'clock. And we are going to be in the other room, in the side room tomorrow, so be there or be square. That session is focused around measurement.

So, today, welcome. And the administrative business, I will get back to that in a second.

We have got berth Hubert presenting on how to make an application fully IPv6 compliant, a short talk about IPv6 only experiences from and /AOU Radovcic and the private contribution. And we've got Tomak who works for ISC, that's mostly testify work where he is seeking operator feedback from and I think so you are all operators, hopefully you'll have some.

And last minutiae addition to our agenda is fixing Bufferbloat with cerowrt, if I pronounce that correctly, by Dave Täht. That's what we're going to do today.
There is a quick glimpse of what we're going to do tomorrow. One of the racy guys, maybe you saw them on Monday, we have got the RIPE Academic Corporation Initiative, we have invited a couple of students to submit talks. One of them was about measuring the effectiveness of happy eyeballs, that was on the topic for the IPv6 WG. We have adopted that one.

Then we have got Vesna talking about the ?? a follow?up of last session. Then we have about an hour's talks about all the IPv6 measurements that are done or should be done, how they fit into policy makers and your business in making decisions on IPv6 deployment or not IPv6 deployment.

And of course any other business, if we have got time left or you think, oh, you missed out on this, this is really important, please give us a ping and we'll try and accommodate you.

So, this is today's run. Well, I have done the welcome. The administrative business, the minutes were cycled on the mailing list and we didn't receive any comments so we consider them final. They should be on the website. Anything else?

Yes, we do have a mailing list. It's IPv6?WG [at] ripe [dot] net, subscribe, feel free. Some people still think that's about policy. No. As long as you mention IPv6, we welcome it to our list, apart from real commercial advertising, but other than that...

So, join the mailing list, feel free to discuss and I'm not going to hold it up any longer and leave the floor to Hubert for his first talk.

BEN HUBBERT: I'm going to do a bit of a boring presentation and it's not for you since you probably know about this already and you care a lot about IPv6. This is for other people. Or maybe just ?? maybe there are one or two things that slipped your mind, like they slipped my mind.

Like David just told me, one of our co?chairs, IPv6 is sort of done by now. It's not really acceptable any more to ship an appcation that does not support IPv6 F you try to sell to any kind of serious Government organisation somewhere in the small lettering it will say you must do IPv6. And people sign that and quite often it's not true. Even when it is true, there is a phenomenon I have seen called IPv6 whitewashing, where people say yeah, my application it supports IPv6. Here it listens on the socket and that's all it does for you. You barely pass the test of I'm doing some IPv6. So then people said well yeah, berth, but what do we need to do? I said well, okay, I wrote blog post that said these are the things over at power DNS encountered as we were for the 7th time pro claiming that we were done implementing IPv6.

So briefly, I founded power DNS, it's a name server and it's in regard wide distribution and I have been telling people that we have been IPv6 ready since 2002 or so and we keep finding small areas where we're not so, that's the origin of this presentation.

The thing I want to see in people's minds is whenever you think my application is IPv6 ready, consult the checklist. Whenever you meet a vendor that says we're IPv6 ready, could you point them towards this presentation and say these are some other things that you might have to think about.

The reason, like I said, I mention this power DNS sort of announced that we finished IPv6 seven times. Recently, the voyage err space craft left the solar system for the ninth time. This is the sort of thing you try not to have in your software.

So, where does the IPv6 sit? In our world, IPv6 is done, or application supported, our operation system is supported and then you try to deploy it in the boring enterprise world and they have a load balancer that does not load balance IPv6. They are instruction protection or prevention device that passes IPv6. That's their way of saying we're doing IPv6. It doesn't actually do, maybe do any scanning. People say well that's fine, your application is IPv6, but our boring as shit HP open view uptime monitoring is still doing IPv4. So, now that IPv6 is done, in its sore, we find all these places where it's not quite done outside of the core.

Areas to look at. Of course the first bullet is when people say we 're doing IPv6. We can BIND to the any address and we're done.

And then it turns out that there are these other things to talk about like converting host names and presentation addresses in some kind of unified format, and determining what it means to connect to a named host with multiple IP addresses. Then there is a curious case of where your IPv6 works sometimes, but not always. And there are some security aspects, filters which I will mention.

The socket, people just open the socket and declare victory. The tricky thing is many applications just binds to the any address. On Linux ?? sorry, one of the things we have to deal with and that we encountered is we said we are IPv6 ready now and then we had some users who said we can no longer deploy your software because we have band IPv6 from our networks. These are are Government organisations that haven't finished writing an IPv6 policy document yet, so they wrote a really small IPv6 policy document that says no. And they demand servers, operating systems, applications that just don't do IPv6. FreeBSD has done a wonderful job at being able to exterminate IPv6 from the system, the code doesn't compile any more even. That's really impressive so.

First thing in supporting IPv6 is supporting ?? not supporting IPv6. Many applications will just randomly create some sockets and may decide not to use them. On such systems, cannot even create an IPv6 socket not even on loop back. So that's something to think about, with able to support not IPv6.

The lazy way of implementing it is listening on the IPv6 any address and saying, well, give me all your IPv4 on IPv6 as well. And that was a nice hack, a nice quick way of doing an implementation in a few letters. The problem with that is that if you try to BIND to the any address for IPv4 and IPv6, on Linux it will collide, on the BSDs, it will not, which is confused the hell out of everybody. The easy solution for that is to tell Linux to behave like FreeBSD, if I want to BIND to IPv6, that's what I mean, I want to BIND to IPv6. It is important to realise this and that it also has some security implications.

Few other notes to think of. You can take the code of an application and put it full of socket addresses for IPv4 and socket addresses for IPv6 and all carry them around in parallel, and that is going to suck in a big way, because in any case where you forget to copy and paste it, you are doomed. It turns out that way back in the 1960s, that's not true, but when they invented the Berkley socket API, they had a grand idea they said, well, we are going to take this address structure and we are going to put the family field where you say if it's IPv4 or IPv6, we are always going to guarantee that it sits in the same place. So you can pass around addresses without telling the operating system which address family it is because it can see that from the address, from the structure. And this allows you, as a programmer, to make a union of two struts, one containing an IPv4 address, one containing an IPv6 address, and pass that union around at a universal address locator. If you decide not to do that and put your code full of if family is IPv4, if family is IPv6, you will misplaces. You will forget them. So, by all means, use this union trick. At the end of the presentation I have an URL that describes this, and it's a neat trick, it works.

However, then you discover on deploying that part of your users cannot use your software any more because this union that carries the IPv4 and IPv6 address is of course longer than the IPv4 address. There are operating systems that like to do a lot of checking. And they say if you supply us with an address that is longer than we are expecting, we are not going to be kind about that, we are going to break your programme. So, use the union and supply the correct length of the address.

This is the part where I also hope to save you pain. If you take an IPv4 address, an IPv4 address strut and you put in the address, the port number and the family, you're done, you can use the address to connect to things. If you take the IPv6 thing, you do the same, it sometimes works, and it sometimes doesn't. It turns out that IPv6 addresses have things called flow info and to this day I'm not sure what a flow info is, except that it needs to be zero. The trick is whenever you use a sock adder for IPv6, just zero it first because no one knows what's in there, just make it zero. This cost us weeks of debugging because we had a guy that says it works if I launch it from here and it doesn't if I launch it from here. That suddenly put something that was non zero in the flow info field. So think of that.

This is not a one that tripped us up. IPv6 has very neat feature link local addresses. My new laser printer, it's wonderful, it supports IPv6 out of the box and the only reason they do that is because I think that some kind of big corporation says we want to have 1,000 of these printers but they must do IPv6. And it's wonderful because even without configuring the printer it has a label on the back with a Mac address and its link local address. I bought this printer the moment I saw that it could do this. And then I fired it up into nigh network and I couldn't connect to it because link local addresses, you can have Mac addresses that appear in multiple segments and on multiple interfaces, so to connect to a link local address you have to provide the link local address and the actual scope ID which means which interface it is on. And you get that ?? on Linux it looks like this, this is how you specify I need to have this link, I need to have this link local address on this link.

The issue with that is that it doesn't actually work everywhere. The Chrome browser is standard compliant, none of the http standards specify that you can add a link ID to an IPv6 address. And not only don't they specify that no one knows how it should be done. So they don't do it.

Then we got a power DNS customer that had the policy of binding all local services to link local addresses and only have some kind of router on the outside dot mapping, so they couldn't run power DNS as it was intended. So, if you support IPv6, make sure that you also support these link local addresses because one day someone will need them.

With the attempt to use the laser, I killed...

Presentation format. This is one thing that we also sort of neglected to specify since 1995, or 96, when they were working on this, if you want to specify an IP address and a port with IPv4 that's simple because we use this colon, which is night, IP dot dot dot column 53, unambiguous. And the IPv6 people also like to colons, so they use all of them. So now we we no longer have a way to specify an IP address and a port combination. This has led people to invent several competing formats for it, and that's bad enough, but pick one, just pick one format for specifying, this is Hoy I part IPv6 addresses with port numbers and this is how I will log them, because we have discovered the hard way that lots of people have scripts and seem systems looking at log files and they need to have a standards for extracting your addresses. So pick any of the three, power DNS picked A. ICS picked C I think. A is what you need to do. A is the standard. So ?? 27, 32, everybody should read it, not many people have. Pick one, because it's hell.

Also, note that whatever sin telecommunications you choose, they suck from using on the command line because all these cache back characters are special.

Is anyone in the room that knows why they picked the colon for IPv6? Dot would have worked for me, but too late now.

Conversion: There is a zoo of ways of converting presentation format, which is what we humans like to read into the actual form. Just pick get adder info, it's the way of converting scope IDs, it does it right. There are other interface that is don't do the scope addresses but it will confuse you and your users. Get add are infor it has an non standard way of returning error messages. On Linux it turns out that by accident it does confirm to what you expect. On FreeBSD they have taken a bit more liberty, so on FreeBSD you will silently ignore errors on IPv6 addresses unless you actually read the man page.

It does not deal with the port number issue for you recollect that's for to you take care of, but it does deal with scope IDs. This is another confusing one. Someone says you need to get your data from access to power DNS dotcom and it has pour and address addresses, which one do we use. We IPv6 want you to use IPv6, because we like that. General users actually don't like that because IPv6 tends to be slower because it's tunnelled, it tends to be less available. The supreme solution that most web browsers are trying to do is to see what works and cache that. My phone knows that it can retrieve e?mail over IPv6. When I leave the house and I'm back on the cellular network which does in the do IPv6, my phone remembers that I could. So, for a brief instant when I leave the house my e?mail is broken. This requires thinking, there is no golden solution for it.

So there are two things. One, document the policy of your allocation, what will it do? Will it prefer IPv4, will it prefer IPv6 or is it something that be configured? And lastly, give the user some way of overriding your choice. Because the user may know better and they may not want to provide a numerical IPv4 address as the only way of telling you to do IPv4.

So, like secure shell does with dash 4, to specify IPv4, something like that needs to be in your configuration file.

Filters. A long time ago I was already enthusiastic about IPv6. This is a really long time ago, 2002, and I applied a patch to Postfix so that it would listen on IPv6. And that batch conveniently did not think about access control lists or who could use my server to relay and some friendly programmer had decided that an IPv6 access rule would match all IPv4 addresses, so overnight my server was turned into a global relay of spam for the world, thus interrupting my holiday.

So, if you do an IPv6 whitewash thing where you say my application listens on IPv6, also ponder what that means for your access control lists, your filtering and all that other stuff, and specifically communicate really well with your helpful programmer that if someone tries to match an IPv6 address to your IPv4 ACL, it should not match. This is actually a philosophical thing but don't assume that it will do the right thing. Or your holiday will be interrupted.

There is also an interesting thing if your user decides to BIND to the any address on IPv6 and support IPv4 which means that IPv4 address also come in mapped to IPv6 addresses. So, if you previously blocked a class of IPv4 addresses, reBIND your servers to IPv6, it might no longer be blocked. So, this is something to preferably make your access control list recognise these mapped IPv6 addresses as secret IPv4 addresses.

Web servers, proxies, forwarders, in those five to ten years, people added lots of features like oh we have a small embedded web server that you can query, you can notify us remotely. We can forward your query somewhere else, and immediately after I wrote the blog post that is the origin of this presentation where I proclaimed for the 7th time that power DNS was IPv6 ready, it was discovered that the built in web server, you could specify that it had to listen on IPv6 but it actually wouldn't. So really go through all your features, all the things because the real issue is if people deploy on a network that needs IPv6, everything has rolled out and then they discover that you don't listen actually on this and that port and that becomes painful. So just go through the list. If you proxy, if you forward, if you filter.

The other thing that we have had is that we ?? power DNS can connect to might SQL and lots of our databases. The policy ?? if people put a name if there, say, connect to this server, then I don't know what my SQL or post SQL will do. So, this is another thing to do if you really want to have IPv6 only configuration that is also part of that resides outside of your own source code, but in your libraries.

This is painful. Many applications store some statistics on remotes that are talking to them, especially in terms of denial of service attacks. For example, who are your top talkers, who are the biggest remotes. And for IPv4 you can get away with just keeping statistics for all your are remotes because they are never going to be more than 2 billion of them. I have more than 2 billion IPv6 addresses at home. So, if you naively port your statistics gathering from IPv4 to IPv6, suddenly everybody will a /64 can just fill your memory and since IPv6 addresses are bigger to, even to begin with what you get away with with IPv4 you don't get away any more with IPv6 and you build a glaring denial of service attack. If you keep any IP remote address, you need to implement pruning, which could you get away with without an IPv4.

Finally, you are not done until the outcome of the command above fits on one screen, and it's a screen that's affordable. If you get a lot of output because this basically says find any time that someone connects a specifically explicitly to IPv4. You should almost ?? you should almost get ?? in an ideal programme you would get one or two hits on this line and the rest would be abstracted away. Even within power DNS, if you do this, it fills a screen, because there are too many places where we still have this sort of no, sir tolling afore IPv4. (Nostalgia) if you are auditing someone's programme and he told you it's fully IPv6 compliant, this is the line to one, and it always finds stuff. So just try to remove the number of lines that it has.

There is a nice trick you can do, and I think some of the presentations after mine will be about that. Just take away IPv4 from your server and try to launch your application. And this will quickly help you find all these places where you BIND to IPv4 but then don't use the sockets or etc., etc., this is a very nice way of ?? and alternatively do the other way around, turn off IPv6 again and try to launch your programme because in general, you also find places where by now you demand that IPv6 is present.

There are two URLs below which are too small to read. For that I have on the final page, there is an URL of this presentation, you can of course also find it on the RIPE site. The blog post has a little bit more detail than this presentation. And the other blog post details how to make a union of an IPv4 and IPv6 address and use that for all your connectivity needs.

Thank you.

AUDIENCE SPEAKER: And /AOU Revenue check owe. I have a couple of comments and one question. So, first comments. You were talking about the policy on what the host name S one of the quarters of the happy eyeballs are a 6555 I suggest people take a look and see the discussion there. Another one about the zone ID the connection, the niceness of the standards, there are so mean to choose from to there is now a standard about specifying the zone ID in the URL. And finally, a question, so all of this stuff, and I related a lot to what you wrote and it's a lot of fun to try and get IPv6 in some kind of code and that's a lot of repetitive work for everyone and I wonder, what do you think that maybe we as a community could co?organise and put a this boring function that is need to be repeated by everyone in some kind of library?

BEN HUBBERT: I think the biggest thing we could do is that this union trick of having one address that can embody both an IPv4 and IPv6 remote, that needs just a little bit of glue to become generically useful. It's not a lot. So this could be the kind of ?? I wouldn't actually have to be a library, but it could be the kind of code snip /*ET that you could publish and say copy this bit and it will help you in your IPv4 to IPv6 migration and you will not have to reinvent the wheel. Because the one thing we want to prevent is people taking their already too long source code and doubling it in length because it's full of if family is 4 then this, if family is 6, do that. That's the smallest thing I can think of.

AUDIENCE SPEAKER: And regarding the code snippets, I will have a little bit of code snippet in my presentation, why I think it should be a library.

AUDIENCE SPEAKER: Benedikt Stockebrand. A couple of comments, first of all about the IPv6 only option. Basically you can have three option. You can say I want to do IPv4 only, I want to do IPv6 only or I choose both of them using the mapped addresses. And of course there is a fourth option, which is taken by default, which is leave it to whatever the default setting in the operating system are a that that can be useful in certain situations but it can really byte you in the ass, so, you want to think about what you need when we work on this. It's an extra step when you deal with sockets or programming at that level, but ?? well you have to do a socket BIND blah blah blah blah, so it's one more step you just have to know about. So the biggest problem here is telling people how to do this properly.

Next thing is about the link global scope. What ?? when you write normal applications, things that belong to the application layer, conceptually and wait they are programmed, you shouldn't really normally touch link local addresses, because as soon as you use link local addresses in applications, what happens is that the application will define the topology we have to use. If I use link local addresses, it falls in the application, the application forces me to put multiple machines on the same network which is something I really want to avoid, some of us are probably old enough to remember what happened then the net buy os days when you had 800 machines in a single ethernet segment because otherwise things wouldn't work and we don't want to go back there again. So, in many cases, I'm actually considered a reasonable move to stay away from link local addresses in normal applications.

Multiple addresses in the DNS on the interfaces and so on. That's pretty much a necessity. Independent of IPv6, IPv6 makes it even more important, and the dual?stacking makes it even more important, but I still remember the times when we had multiple data centres with multiple web servers with a sharing ?? who were sharing DNS entries, so we had eight different IP addresses for www.example.com, and we were dealing with web browsers that couldn't do that because they were stuck with this stupid host by name thing that could only return a single address. So yes it puts a bit more complication in there but we actually need this sort of functionality to some degree.

And one last thing on ACLs. It's a bit of a hack, and it's potentially painful, but sometimes you can just work around this using a local packet filter instead of application related ACLs. It's not necessarily nice, but it does the job and you get the result you actually want. That's it.

CHAIR: Do you want to respond to that quickly Bert?

BEN HUBBERT: There is one thing on the link local stuff, I would tend to agree, but in the real world there are people that have big standards that says this is what we do here. And I like my software to be soft of opinion ateed to make it easy to do the right thing but it generally is not advisable to, if you disagree with something to make it impossible, so it is something that it has to be there.

AUDIENCE SPEAKER: It's a point, but the problem is if you put things there you might get people going the wrong tack to it's one or another.

BEN HUBBERT: I would try to stay away from computing.

CHAIR: Thank you Benedikt. I'll take these two last questions.

AUDIENCE SPEAKER: Dan. First I'd like to say thank you for bringing this presentation here. It's an important part, certainly we've seen in looking at moving IPv6 deployment forward and getting applications out there. I would again second what was said up at the front mike too around the happy eyeballs are C 6555 and there are also some other drafts coming out around happy eyeballs for few, where people are looking at this mechanism of testing both and figuring out which one is fastest.

The other piece, back to your original question around representing the IP address and the port number I think somebody might have called it out, but there is a specific RFC 5952 that says that you should use the brackets with the colon, so choice A is now made I think by most folks out there. So, again, thanks for putting this out there.

BEN HUBBERT: It also may be God to know that Dan York wrote the book.

AUDIENCE SPEAKER: I would like to comment about the v6 only, it's not kind of default behaviour, it's actually specified by RFC basic circuits, and the behaviour that you get both IPv6 and IPv4 is standard behaviour. If you configure your OS to ?? you are not compliant on your applications and some applications may assume that.

BEN HUBBERT: I agree that we, after many years of having that position, we changed to defaulting to the BSD behaviour because we had so many reports of people taking a working configuration from FreeBSD and moving it to Linux and no longer having it working and the other way around. I agree with you that it's a policy that can be set but it turned out to be really confusing for people. So that's why we now say look, this is what we do.

CHAIR: Okay. Well, thank you Bert, and as I forgot to say in my introduction, many thanks for taking up the challenge because I read his blog and I thought it was interesting and talked him into doing this talk to he converted it into slides. And thank you for that. I'm sure you'll hang around for more questions. Thank you.

(Applause)

Next up then is Andrew Yourtchenko who is going to show a bit of what happens if you don't listen to Peter's talk.

ANDREW YOURTCHENKO: So every presentation has a story with a happy ending. This presentation doesn't, and I'm presenting it as an Internet citizen without any hat.

So before we go to client experience, let's talk about dual stack network. So as you deploy both IPv4 and IPv6, while the network from the client's stand point it generally stands on its own. Why? Because if he everything is done correctly everywhere, then IPv4 and IPv6 kind of mask any imperfections in each other. Which is a good thing. Because the end customers don't care whether it comes over IPv4, IPv6, or other carriers, they just want their applications to work.

Now, of course, if we keep doing dual stack forever, then why do we do IPv6 in the first place? So we need to remove IPv4 at some point. So, how do we get to bike without the training wheels? And if we start looking from the content stand point, we move from the content eyeballs, content is going to have to stay dual stack for as long as there is anything that has IPv4 and luckily, while content is relatively simple.

Almost the same thing about transit, and I have the asterisk here because transit on IPv6 and that I can say is the end user monitoring... in various networks that latency is not the same over IPv4 and the IPv6 and the differences in latency over time are also not the same.

And that's where I will end my rant about the transit. So you all know about it.

But the important part to talk about the eyeballs, so the device that is connect to the network on the edges. So, that's where we need to start thinking about turning off IPv4. And in order to be able to do that, we need to have, on the order of magnitude of 10% of our users be able to use IPv6 only in order to be taken seriously.

So, as I said, I do, as part of my activities, do I various hands?on in the event light networks. I have the data from those networks and that's something that I wanted to present here is just to backup and contribute to what I'm going to say.

So that's just the hardware distribution and the wi?fi networks that I run. It's a typical network and various different events, various different gatherings, it's different by some fractions of percent, but the ballpark is the same. So, 40 something percent Apple, 40 something percent Intel, well, a bit less, and the rest.

So, now let's take a look at the software side. All right, so what distribution do we have operating systemwise?

If we talk about the laptops, this is all fairly well engineered, fairly well understood. It's with some tweaks, it works on IPv6 only. So, talking about tweaks, if you connect any more stack to the network without the DHCP 4 mechanism then the device will eventually assign IPv4 local link address. Then the RFC describing the IPv4 local link addresses says that you need to proxy Arp for everything if you don't have the global address, if you want to talk to a global IPv4 address on the Internet. So, this results in the applications that think that they can communicate with v4 hosts and they try to do that. So, on the network without IPv4 you don't have IPv4 but you have a lot of Arp. And the URL here is the draft that I submitted just in time for this meeting, so actually just to be able to put the URL for this slide and I will socialise that with the Working Group in the IETF who has turning off IPv4 explicitly as one of the work group items and they have I think other draft in the work where the server will explicitly tell from the DHCP to turn the stack off and this is just one other solution, so we'll see if at least one of them sees the light of the day.

Now, that's all relatively not so sad so far. But how many people in the room has a smartphone or a tablet on the network? That's almost everyone. So, none of your devices will be able to work on v6 only. Pretty much, well minus some smaller fraction.

So, and here I will go a little bit into specifics. So, Android on the IPv6 only which Wi?Fi. The reason for that is that it doesn't support DHCP 6 nor rDNS S. What do you do with the IPv6 address? Nothing. If you own an IOS device that's a little bit better. But the danger is in the details. The devil is in the details, which was in this morning's presentation. Because IOS 6 actually fixed a lot of stuff that is related to the stack. So, you can get the Wi?Fi icon once you connect to IPv6 Wi?Fi. Fantastic, except at the same time, the folks who deal with the usability said hey, well, it would be great if the application checked whether the Internet is available or not. And if the Internet is not available, it would prevent any activities trying to connect or trying to time?out and so on and so forth. And that was also great. Except that the code snippet which was published has IPv4 Internet connectivity check. So, if I'm sitting on the IPv6 only Internet, even behind a NAT 64, I get the Wi?Fi icon but I don't get the Internet, which is kind of sad.

So if you are developing the app for IOS, please, please do the check correctly, so don't take the example from the website and just cut and paste it.

Can we make this work? Yes, so we can distribute fade, 164 sub?net, we can have local recursive resolver, and, well, everyone upstream will think that is kind of IPv6. Is this IPv6 only? Well it's at best IPv five and a half.

The nice part that actually makes things work, and it's fairly obvious, I had a lot of hesitation in putting this slide because it is ugly, but I think this is an important part in the path towards being able to cycle without training wheels. And what I would like you to take away as one message from my presentation is that please, please, please establish your own learning bike without training wheel programme, and talk to users, study your traffic, study their devices, and try to understand what does it take to get one, 2, 5 and finally 10% of your users to use IPv6 only network and then talk to vendors where needed, open blog reports and let's fix this. So ?? and I hope that some day I will be able to present this and say that the story has a happy ending. Thank you.

CHAIR: Any quick questions for Andrew? No?

AUDIENCE SPEAKER: First of all, thank you. I have no questions, but mostly comment. I think it's a really, really great experiment and we need more like this to know, to let us know that Android doesn't work proper on IPv6 only, so know what else could be broken. And find probably a better transition and so on.

Thanks a lot. From ?? I'm talking Internet citizen. And we can discuss off line the Android problem.

ANDREW YOURTCHENKO: One thing I forget to say is on the non Wi?Fi is actually works. There is good news as well.

CHAIR: I'm not going to try and open a huge can of worms about wireless at conferences and stability over it, but the thought we had and it crossed our mind before, maybe we should create and IPv6 only network at the conference just for people to suffer, and just a quick poll, a show of hands, oh yes, says bring up an IPv6 only. Cool, that's about half. Who says that no, do not risk the connectivity? Ah, Serge, I think you are overruled here, any comments? We'll bring it up again and see if we can find a way to not break the connectivity and yet provide you with IPv6 only connectivity. Thank you Andrew.

Which is a nice bridge to our next talk, waiting for DHCP 6, so here is Tomek talking here to fill in the gaps.

TOMEK MRUGALSKI: Hello everyone. My name is Tomek Mrugalski. So I'd like to talk a bit about the latest developments in DHCP 6.

Okay. So the, I'd like to present a couple of DHCP 6 mechanisms that are currently being developed in IETF, so in particular, I'd like to present a client Mac address option, load balancing, the concept of DHCP 6 fail over, some issues related to provisioning IPv4 hosts in IPv6 only network, and briefly discuss radius option.

Okay. So, a bit about me. This is my first RIPE Meeting, so why am I even allowed to talk about DHCP 6? So, I have been involved in DHCP 6 development for the last ten years. So, I did my masters and Ph.D. in DHCP 6 in 2003, so it's been under development for the last 2010 years. I have worked some of the years for Intel, in the last two years I have spent at ISC working on a new DHCP 4 and DHCP 6 implementation and I am also active in IETF since 2009 and I have been appointed DHC co?chair about two months ago.

So, some information about ISC. ISC is a non?profit corporation that is focused on a couple of things: Developing production quality open source software, developing protocols and operations, so I'd like to talk about protocols.

So this slide summarised all the active work in DHCP 4 in IETF.

So, the first topic that I'd like to present is the issue of Mac addresses, or Internet link local addresses in DHCPv6. So in DHCPv4 it was natural that there was a mapping between IPv4 addresses and Mac, because DHCPv4 protocol uses Mac field directly. So in DHCPv6, there was a new concept of DUID introduced. The idea was to have something that is more stable, so it is generated once and then stalled, so, in v4 the issue was that when you have a device like, for example, a regular PC and you change the interface ?? the network interface card, then you have a new Mac address from the DHCP perspective, this is a completely new device. So, the idea was to solve this problem with the DUID. So currently there are four different types of DUID defined link local, link local enterprise base and UU I do it. So the original issue was almost solved, but it introduced new problems. For example, if you have dual bolt machine, if you work under Windows, the DUID is generated once. When you then reboot, it tells Linux for the first time, a different DUID is generated because the default time is link lock?out plus time and of course there will be generated at different times.

Another issue is that when you have one virtual machine, you'll boot it up, you'll set it up, everything is working as you expected and then you clone it like ten times and then suddenly all the machines have the same DUID. So...

And due to the nature of DHCPv6 protocol, the Mac address information was not available in DHCPv6. So, the problem is that for the directly connected clients, the server could get the Mac address. However, if there are relays and of course there are in most networks, there are relays, that information was not available. So, fortunately, that problem was well understood and there is a draft that was ?? when I was preparing the presentation, it was in RFC ?? it was published 16 hours ago, it's officially RFC 6939, so the idea is that the relay will insert that information about client Mac address, and will send this to the server. So from the server perspective, it will be possible to define static mapping between Mac address and assigned IPv6 address.

So, the other mechanism that I'd like to talk about is load balancing in the DHCPv6. So, the problem space is ?? so, of course it's beneficial if you can deploy more than one DHCP server on a given link. So, the base protocol allows you to do so, and it's possible to define a preference option. So we'll say that this server is more preferred, the other is less preferred. But that doesn't solve the problem, because the server that is more preferred will get all the clients and they will serve them. Of course you can configure the preference to the same value, but then still you will have the problem that the faster server will respond faster and most clients will get that, will get to that server. So, the solution that was proposed is load balancing.

So, the idea is that you deploy your two servers, two or more servers, and for each incoming packet, the server calculates hash value of the client ID and there are 256 packets and depending on how you configure the servers, it will be one of the servers that will be responding.

So in this example, the first server is configured to serve the bottom half, and the second server is configured to handle the upper part. So, this is an example how you can split the DHCP traffic between two servers, but the nice thing about this approach is that you can split it into many different servers if you want to.

So, it's good for load balancing. However, it's not suitable for things like high availability. So if you set up those two servers in this way, one of the servers falls over, the clients that were served by this server, they will need to obtain a new lease from the other servers. This solution does not provide link stability. And it does not offer redundancy.

So, for the two issues that I have mentioned. The ultimate solution is to define DHCPv6 failover. So there are many people who say, okay, with v6 we don't need neetd fail over because there are so many IPv6 addresses. So the issue with this approach is that when your server changes, the address changes, so you don't have lease stability, so for clients that require long lasting connections, this is an issue.

And also the idea that we have so many v6 addresses that it's not an issue. So there is the problem with prefix delegation and prefix delegation, suddenly we don't have that many leases.

So the other myth about failover is that we have many databases and they will save the day. So here is a simple example. So, let's configure a single sub?net with just one lease. So we have two servers connected to two databases that together form a multi?master database. So if there is a network split between them, and they are two clients connecting to first server, so the server only has one lease, so it will assign it to the client. There is a second client connecting to the second server, the second server also has one lease, so it will assign that lease as well. So this the breakdown because the same lease was assigned to two different clients.

Of course, having sub?net with just one lease, it's not realistic, but if you have millions of addresses you just decree the possibility, then you solve the problem.

And of course you can repeat the whole exercise with preference delegation, then you will have added bonus of routing issues. Of course there are people who say that no, but do you believe Kate added detection will save me. Again it won't work for prefix delegation.

So, failover is a complicated process. So, with the DHCPv4 failover, the standardisation was going for for 6 years and ultimately failed in 2003. The reason was that the draft was basically too hard, it was 130 pages long and it was very difficult to read. So with v6 we decided to try to split the problem into smaller pieces, so we split this into four steps. So the step zero is the preparation. It talks about reading the considerations, it was published recently as RFC 6853 or BCP 180. So the next document that I very much encourage you to read and comment on is requirement document. It explains what is expected from the failover.

There is a second document that is available that fulfils all the requirements that were defined in here. The second document does not provide implementation details, so this is something that will be a viable in the third draft.

So, very quick overview of the failover itself. So it's based on v4 failover. It's a hot stand by. There is no load balancing because this is provided in a different way. The first recovery from two different issues, the server crash and network partition.

Okay. So, another activity that is quite active in IETF is discussion about provisioning of IPv4 nodes in IPv6 only network. So there are many, many different approaches and there is a discussion about which of them is the best and how to provision them, how to provision those different solutions. Of course using DHCP.

So, the last draft or mechanism that I'd like to talk about is radius option. So, this is basically v6 version of a mechanism that is available in v4. So, the idea is that when you have a network access server, so your clients connect via this relay, that's a relay from the DHCP perspective, so it asks the radius server to get parameters and basically do access control, and so the idea is that those parameters can be then sent to the DHCPv6 server and the server itself can get those parameters and use it depending on the allocation policy, assign the extra options or do different things.

So, there are a couple of the drafts about DHCPv6 stateful issues and the differences in how the DHCPv6 servers assign addresses and prefix delegations. There is quite large design team formed to tackle the problem of multiple provisioning domains, there was ?? and of course there was everyone's favourite topic, the routing configuration over DHCPv6, so it was very long and painful and very heated discussion, but it seems that the idea is dying slowly.

Okay. So, what I'd like to ask you to do, is if you have opinions as operators, if those mechanisms will work for you, so please tell me now, that's good. Even better will be if you post your comments to the DHC mailing list. Of course for you it will probably be yet another mailing list, so it would be sufficient to subscribe both your comments and then subscribe. I would be very happy if you stay subscribed and participated in the suggestion.

And of course you can consider going to IETF meeting, the next one is in Berlin, so it's quite close.

And I encourage you to go to the DHC Working Group home page.

Okay. So that's it. Any comments, questions?

CHAIR: While people rush to the microphone, you can tell you you can add one of the DUID things to your list when building a network which included doing prefix delegation to CPE, swapping the CPE does not release your prefix delegation and we ran into that operational problem a few times in the DSL network because the prefix delegation is bound to the DUID and that's tied to the CPE. So if you to say away the CPE because it breaks down you have to wait for your lease time to expire.

AUDIENCE SPEAKER: Shane Kerr. I am a colleague of Tomek's.

A question: What's your feeling for kind of the typical time between when a new technology gets standardised in DHCP land before it actually enters products?

TOMEK MRUGALSKI: It's very much different depending on the nature of the technology. For example the option that was standardised about DS?Lite configuration, that's worked for like 50 minutes of working, and would failover it's about a year of implementation. So, it's very much different. And of course it depends on how the project is managed and how many people are assigned, so...

SHANE KERR: Yeah, you need more people.

AUDIENCE SPEAKER: Paul Ebersman. On the new relay option with getting the hardware address, is that in the IFC road map for getting the server to be able to consume that information and put it into the lease?

TOMEK MRUGALSKI: That basically, how the information is used is up to the server allegation policy. One of the uses for this is that you can log it, so there are in various countries that are legal requirements that you ?? when the law enforcement comes to you and say hey, who was doing some bad stuff from this address? You are supposed to provide Mac address, so this is one of the used cases. And the other one is that depending on how the server is configured and what the server is capable, you can basically assign addresses, prefixes and different options based on that information. And of course you can do access control, for example you may maintain a blacklist for specific addresses or a white list that you want to serve only known clients.

AUDIENCE SPEAKER: Yeah, plenty of used case and I actually have a number customers who are quite interested. I am more curious, there are two requirements. One is that we have to have relay code that actually inserts that information into the bit so we need to start beating obviously on people like Cisco and Juniper, but we also need the DHCP server to actually get that information in the packet and then put it into the lease or somewhere, I was curious that that's in the plans for ISC's code for their server?

TOMEK MRUGALSKI: I cannot confirm or deny this. So, yeah, but we're thinking about this. So... you had Benedikt Stockebrand. I don't quite understand why you worry about scaleability, because the obvious approach to me is really to put up several DHCP servers and split the assorted sub nets over them so I don't need to do any sort of active cluster, whatever, sort of stuff. But I'd rather go with the simple straightforward approach, set up a suitably sized DHCP server and if that can't handle the full bunch of sub nets I have, then I split the sub nets over multiple instances. Did I miss something there or...

TOMEK MRUGALSKI: No, I am not saying that you absolutely have to deploy the load balancing or failover in every single network. Of course if you have traffic that is, that your server is big enough to support all the traffic, that's fine, this is one of the approaches. If you don't, then that's also fine.

AUDIENCE SPEAKER: The problem I see here is as soon as I deploy some sort of cluster, whatever mechanism, I at some significant complexity which has a nasty tendency to form people's feed. So, in my opinion, this approach of providing increased scaleability on the service side has a significant draw back if people actually try to use it in a production environment. That's what I'm sort of worried about.

TOMEK MRUGALSKI: Yes, I agree completely. So the second draft that was presented, the DHCPv6 failover requirements, there is a section in the draft that specifically talks about cases when the failover is not for you, so, to kind of disperse different misconceptions that, because many people say yes, yes, of course I need failover, and sometimes when you investigate that problem, in many cases you don't need failover, so I consider failover as the last resort because everything else is much simpler to deploy and maintain.

AUDIENCE SPEAKER: That's not my point. My point is not about failover or high availability. My point is about high performance clustering where you have multiple servers to share the load. And in my opinion, DHCP is one of those areas where simply you don't need that because you can scale by assigning different sub nets to different DHCP servers. I'm not sure if ?? if I have 10,000 sub nets and I have a single DHCP server who can't cope with the job, the obvious approach to me was to split up those 10,000 sub nets into 5,000, have two different DHCP servers, or high availability clusters of DHCP servers, and then assign those, the first 5,000 to one DHCP server and the next 5,000 to the second DHCP server and that would solve the scaleability issue without any extra overhead complexity or whatever in the protocol.

TOMEK MRUGALSKI: Okay. So, as many other DHCP extensions, these are not mandatory features, if you don't want to use them in your network, that's fine.

AUDIENCE SPEAKER: Well, the point is really, once you put it into the specs, people will have something they have to configure or worry about and they will make mistakes. If it's not in the specs, it's not implemented anywhere hopefully and it will make life easier for everybody else.

TOMEK MRUGALSKI: Yes, but there is a difference between saying I don't need it and nobody elses is needed, so ?? and of course I think this is mostly the implementation issue. So I expect that every implementer, myself included, that will implement this, it will be disabled by default, so, unless you explicitly do certain steps to enable it, it won't be active.

CHAIR: Can I ask you, just ?? join the IETF and discuss this on the list, because it sort of comes back to implementation.

Well, thank you, Tomek, for the presentation.

(Applause.)

Which brings us nicely on schedule, we have got 20 minutes and one presenter left, that will take us up to the coffee break, so, Dave, who will be next, and who is now making his way forward. This was a last minute addition. As always, some stuff comes up really late, to be honest it didn't really come up really late, we sort of lost track of an e?mail, I have to admit. So it was a late addition from our side but not from Dave. As always, by the way, we are looking into agenda items for RIPE 67. So if you have got any follow?up talks for this or work to be done, new policies, new standardisation things you want to discuss, please do send us an e?mail

DAVE TÄHT: .I'm Dave. I run this with Jim Geddes, I call that lately because when I started it with Jim, I was afraid that the congestion problems we were seeing on the Internet and Wi?Fi and wireless were going to cause collapse and I was also very concerned that it would spread to the ethernet side and I was even more concerned that we were not going to be able to deploy IPv6.

So we sat down and started coming up with ways to work on the problem. So, as I stand here today, I am pretty confident that the Bufferbloat issue has been fixed. We have been analysing the solutions presented to it over the last two years, the kernel code landed in August of last year and is now spreading everywhere.

The initial slide I have here shows that users are typically seeing under load latencies around the second using the, we have managed to cut that by two orders of magnitude for many forms of traffic.

So, about two years ago things looked really dark. We needed to attack the Bufferbloat problem at almost every level and in particular, we needed to know that the algorithm that is we were looking at were implementable and deployable on consumer hardware and edge devices and we needed to know why IPv6 couldn't be deployed and why all of our systems were showing such latencies and problems, and why did home gateways suck so badly?

We tore apart a whole bunch of consumer equipment. Dumped the flashes, and decoded what stuff was so on and so forth and we discovered that even brand new hardware shipped to End Users was typically using five plus year old Linux kernels and even older packages. We would go back and talk to vendors like that and they say we don't have any margins and nobody is asking us us for IPv6. Did you ask them in detail what their code looked like, they didn't know, they were just using old ports by the vendors, okay. We went around talking to retail vendors and academia and the ISPs and everybody was pointing fingers at each other, gave a great talk about it right around that time answered said the vendors have to fix T the vendors weren't working on good margins.

The worse thing that we found though as we were working on it was that binary blobs were everywhere, they were embedded in DSL, mode items really close to the modem, they were embedded in wi?Fi and as soon as you had this, your ability to do remote updates goes out the window. So the result was grid lock, five year old kernels, no way to make any progress forward.

Yesterday I was watching one of the presentations given here that talked about the DNS amplification attack problem. And it was all about oh, we have to practice BC 38, BCP 38, but it didn't it struck me, I said wow, number 2 is a tool called DNS mask, a caching DNS forward errand a version that was released ten years ago. And if you add up all the other DNS masks instances here, it turns out that DNS mask was the largest DNS relay amplification attack source out there. This does not make the author happy. His software does not have this problem out of the box and it also in relation to the fact that DNS mask has roughly a billion users right now.

So how do you get upgrades, needed upgrades out there? How do you fix end user CPE? These are systems that are flashed once and never up grated currently. It's a really big problem that I would like people to be addressing.

So, we started cerowrt. We found that in terms of software, there was no way we'd be able to get all the fixes and upgrades that we needed unless we stayed really close to the main Linux kernel and leveraged their work on the current server space and DNSSEC top space. The second problem we ran into is oh can you support hardware X? No, we don't have time for that.

We went looking for one platform that had all open source software in in it to be able to inspect deeply every single piece it have down to the registers themselves and existing code. And then we said okay, it's Jim and I, coming up with this idea, and we really wanted to also find something that would appeal to hackers and End Users that cared about running things like IPv6. And so we started layering on some features. We have had a lot of people give feedback oh you could never run something like BIND on a home router, we had BIND 9 running on this thing in April of that year, it worked with DNSSEC. And so, we wanted to be able to feed that and we went looking around at all sorts of other operating systems, opportunities and pretty much landed on open WRT.

So, we are still dealing with the social problem and the business and the marketing problem and we wanted to tackle the technical problem but still today, how do you get somebody who has a margin of maybe $2 on their hardware to make sure that they are doing updates continuously. No one has got a subscription model for hardware.

Secondly, we wanted to tackle the Bufferbloat problem which was a rather difficult theoretical problem that's plagued the Internet for about 30 years and we thought we wanted experiment with various other things that are popular in the IETF. We pretty much proven to ourselves that everyone bridging wireless and wired interfaces particularly when you have 802 and 11 G with gig E ethernet, didn't work. So, how do you route stuff in the home rather than bridge it?

And if you are going to route, oh can grandma route? I don't know. And how was it so hard to get working IPv6 when we started in 2011 there was no consumer Rome router that had a reasonable IPv6 implementation. And multihoming is kind of cool. And once you get IPv6, how many people here can remember their IPv6 address? I hate you...

We felt that getting better naming into IPv6 would help.

One of the other things that I noticed, living in Nicaragua, working on child project and related stuff, the Internet still goes down for some people. Will your network still work when the Internet is down. And of course this ongoing problems in all kinds of levels of security and home routers in general.

We said what have, we got a cool platform, we tackled all these interesting ideas and we convinced people like, I don't know, David Reed and Eric Raymond and Paul Vixey and people like that to help, and we sat down and went in defeat in detail for absolutely every problem that we encountered and then we spun really, really fast and fed the results back up to open WRT, back up to the kernel back up to the package maintainers and keep spinning new fixes as fast as we could. And what would happen if we got lots of people excited about doing that? And if we succeeded in fixing Bufferbloat in particular, speeding up the Internet, well, maybe we'd also get a free and cool IPv6 stack with it, maybe some security updates, maybe the ability to remote firmware updates.

Now, I have another 20 slides talking about Bufferbloat in general and the fix called FQ column, but instead I have videos up from my talks in MIT an Stanford and the University of Modena, the Gathering, I have links to that later on in the talk. But we have managed to take a typical cable modem which used to exhibit latency as well in access to a second and cut it down to zero under a rather heavy duty test with 4 up and 4 down streams. We have taken the same algorithm and applied it to Wi?Fi, saturating completely where it used to go up to about 8 hundred milliseconds in this particular test down to 15.

So, a net result of the cerowrt project, the ability to go and test and spin and play with this stuff and the focus on Bufferbloat has given us a really brand new algorithm, something new under the Sun that can speed up the Internet by a lot. Reducing round trip time by a lot. And it doesn't just apply to home gateways. It pretty much applies rate limited anything.

Load balancers, servers in general, servers, DNSSEC tops, home routers and SEPs. We are playing with it, it was really cool to have one platform, cerowrt, to play with it. So, rather than do that whole talk, because I only have seven minutes left I'd recommend you have a look at this video and various and sundry other materials.

But today of the first time anybody had ever asked us how we managed to solve a 30 year old problem in under two years, actually under a year we spent the last year proving it out.

So we picked a cheap router which we liked, that had entirely open source firmware to be able to inspect every portion of it. Then we picked open WRT off?the?shelf and we started to work closey with the open WRT developers to follow the kernels as they came out, feed fixes back in. And we kept adding in things like test tools, so we'd be able to diagnose the problems that we had. And we tested fully with IPv6 throughout all of our testing, it was a first class object and we did things like having poll /POE as a web proxy to do IPv6 versus IPv4 translation. We worked at the driver level to make sure the drivers were good and we worked at the kernel level to smash buffering throughout the stack, and we played around with routing, not bridging, to see if we could make wireless behave better. We had a lot of people say you can't possibly run IPv6 in a small home consumer router. Well, the current build is 8.9 meg bites and that includes about 2 MEGS of tools.

And although I am very happy with assert, it is a research project, the tools and products go upstream where I hope people would productise them, if you want to play with cerowrt, please don't expose your women and children to it.

Along the way we also established IPv6 test sites all over the place. ISC was very helpful, and during this project I had a chance to sleep on Paul Vixeys couch in his office and in Eric's basement. I had a place that had no other Wi?Fi signals at all in California and call it the YURT lab. Over at Comcast we have been testing DHCP PD and we had several predominant supporter, people would go and make sure 6to4 works, 6 in 4 works and all kind of formats worked and then we had these test sites set up that people could use.

So, in this process, as I have been saying, IPv6 was always there. It's really ?? and yes we did turn off IPv4 and cope with that. And along the way over the last two years we have helped smash many bugs in encapsulation, there was a couple of bugs in ECN that were nasty, this particular hardware it would give you an instruction trap on certain kind of IPv6 packets which were slowed performance by a factor of two. We fixed that. There is all kind of routing bugs in IPv6 ?? in general, we fixed. And we worked on all these different encapsulation methods. Not so much cerowrt, but we inspired some people to go out and try solve other problems, so there is a core open WRT developer which has produced a DHCP client which fits into less than 30 K and we have tracked the RFCs and by August of last year, maybe September, IPv6 was working really well on a home router, and tracking the kernel really close. And as of last month, after all this stuff has landed open WRT has enabled IPv6 by default. Open WRT is used by at love things, I don't know how many there are, but it's out there. So, also DD worth and also buffalo's products.

So, by being able to spin so fast, I was able to take the NS 2 code that Ivan Jacobsen and Cathy Nichols presented and I got it into into a main line router in a week. It doesn't happen everyday. Everyone is pushing here towards getting things that that can five years out. And we have spent the rest of the time analysing it. Very happy with IPv6. We proved DNSSEC could work and so on. One of my favourite bug reports, was this one. You are working on devolvement bug E code, cool. I got this bug report about a year?and?a?half ago. And I had an uptime of 259 days and felt like complaining about it. So, he happened to point tout that it happened to him before on a different OS, and unfortunately, he promised to tell me in October 6th and I was waiting October 6th to tell me if it crashed again, it didn't. It crashed again 330 days later. So, there has been two new DHCPv6 implementations, I already mentioned. Also DNS mask has become a full?fledged RAD BD replacement. It has a wonderful implementation of DHCPv4 to slack naming, which really makes finally ties closely together naming and IPv6. Fire walling is better, I can't remember the core FCs number but it's like within one feature of being a full RFC something home router compliant. The only thing really missing from open WRT at this point it a good shaper. I have several in cerowrt, we are still evaluating them. And there is way more stuff.

So, if you are looking to play with some pieces of the future, you can find it there. However, as of last week absolutely everything that was important and ended up in open WRT, so I don't need to work on that any more.

Going forward, Simon Kelly has got funded to DNSSEC, this is a possibility of seeing DNSSEC running on a billion machines. Hope you guys are ready. I am if I had length ?? we pretty much have figured out how to fix Linux Wi?Fi. Cisco is playing with another A Q M similar to cod he will which will also fix delays. We are trying to ?? there is still a few open research issues regarding the new A Q Ms, but whatever. The RRUL, realtime response underload, it's there. I care very much about hardening home routers against attacks. There is a bunch of ways to do that. Research continues. A big topic for the last three weeks has been doing IPv6 source and destination based routing. It looks like Linux's only kernel support is not quite good enough to do it right and we'd like to come up a better API. And MD NS EXT. My big question is what else do I need? Try to run this for me and tell me what I missed.

Go back to my Bufferbloat. This is where you people are today under the kind of loads that are for web pages while you are under a single TCP flow. This is where you could be. This is 12 seconds, this is almost completely impervious to the existing route on this system. A great deal of system of volunteers have worked on this project. Really grateful that they all helped.

And particularly grateful for Tom Sawyer, his inspiration on how to get people excited about painting a fence was very helpful along the way.

So that's it.

Questions...

CHAIR: Any questions for Dave?

AUDIENCE SPEAKER: Hi, I am Vesna speaking for myself as an Internet citizen. I really like your project, I think it's cool and I would like to help you to do expose it to more women and children.

DAVE TÄHT: Thank you. I am sorry, you can't play games right now ?? no... just make sure they're ready. Cool.

CHAIR: Anything else as Dave said, what else is needed? A long list of stuff.

DAVE TÄHT: What else is needed in a home router?

AUDIENCE SPEAKER: Hello. Really good work, really like it. However, I am specifying CPEs for our company right now and how should I tell them you need to consider this?

DAVE TÄHT: Well, this is a hard question. One of the things I would like people to do is look deeply into what's in open WC, it which is only about, 8 Linux kernel and about 8 demons and if you specify and say I would like to have DNSmasq 2.66 or later, that would get you a large portion of the way there for other pieces. So if you can take a deeper look at saying I want these packages and this stuff then you have a much better chance of getting what you want.

AUDIENCE SPEAKER: I'll refer them to your website and say we need this.

DAVE TÄHT: Okay. Again, we have a mailing list. I'd be glad to help next.

AUDIENCE SPEAKER: Hello. I am wondering do you happen to have in your road map plans to accept tools that do active measurement tests.

DAVE TÄHT: Absolutely. Cerowrt ?? if you ever heard of business mark project. I worked a little bit on that one at the same time. Business mark wanted to go diagnose the net and I didn't trust anything underneath the stack to actually give me an accurate test. So, I'm at the point where I really trust the underlying tack and I'll gladly add more tools. I have a huge list of tools that are already built for, it things like Scamper and Netperf and Iperf, but if you have any suggestions, contact me later.

AUDIENCE SPEAKER: I'll ask one more question. I see that RIPE Atlas is also using a heart wear which is now open WRT compatible, I think they use 3020 M, so do you have any plans to collaborate together?

DAVE TÄHT: So, my plan is to keep working on the stuff I truly trust and know and pushing the theory forward. It's not a whole lot of people on it, so we push everything into open WRT and it pushes down into there. So it would be my hope that they are tracking current work in open WRT and I am always glad to help, but adopting a different hardware platform would take a lot of time right now. I'm rooting for them, I am glad to help...

CHAIR: Okay. Well if there are no further questions, then thank you Dave, and that is concluding our session nicely in time. We ran two minutes into your coffee break, sorry for that we had some technical difficulties with the remote. Thank you for being here. Thank you all for your contributions asking questions, and after the break here in this room, NCC Services and the IPv6 party will continue tomorrow morning at 9 o'clock in the room next door. So be there and thank you all for being here and see you in the mailing list. Thanks.

(Coffee break)