OTTD Unlimited?

[Quark]

How about to pack hash from version string instead?

Then how would you be able to tell the version the client needs to get to be able to play on that server?

Pack all know versions to code or some database file and send unknown verison string with newgrf files list from server by additional request.

[Quark]

How about to automatically download grf files just as OTTD downloads world map?

[Brianetta]

And yes, the version string might be short, but it has to be packed into the PACKET_UDP_SERVER_RESPONSE packet, which is nearly reaching the maximum size for a packet.

The version string is the least of your worries. Every newgrf added to the game entails more overhead than the current version string, in that very same packet. 20 bytes per newgrf blows the packet size to five times what it used to be on my server with 15 of the things. Perhaps that's why so many people can't see the server details in the multiplayer dialogue!

Quark, you can't assume that the server or the player has a license to copy any newgrf. Also, your idea for a version database would be unweildy, since there have been nearly 8,000 versions.

[Quark]

Maybe it is possible to ask grf authors for permission to automatic grf transferring from game server?

And database can be self-learning — ask server string, description, grf config, rules, etc. one time and then save it in db (no need to more ask).
8000 * (15 bytes + hash) ≈ from 152 to 296 kb. And it is possible to use first bit of hash to mark that hash is just plain text. So database is not needed to store this 8000 revisions.

[Mr. Wednesday]

Is it better for OTTD to assume that the server does not have permission to distribute the grf, or is it better to put the onus on the server to only operate with a grf that they do have permission to distribute?

(Just throwing it out there, I don't have a firm opinion.)

[Brianetta]

Is it better for OTTD to assume that the server does not have permission to distribute the grf, or is it better to put the onus on the server to only operate with a grf that they do have permission to distribute?

(Just throwing it out there, I don't have a firm opinion.)

That's a difficult one. The devs won't be responsible for which grfs are distributed, but are they responsible for making it easy for others to distribute them? The Napster scenario.

I operate a server exclusively with newgrfs which I have permission to distribute, either explicitly to me from the copyright holder, or generally, through an accompanying license.

I know from experience of administering a server, though, that this is far from the normal. Players continually request specific newgrfs, and many server owners add them without a thought for licensing. People are people, and often aren't concerned with issues like copyright or intellectual property.

The two most popular station sets on servers are Michael Blunck's passenger stations and the industrial stations by norfolksouthern37 et al. The former requires the inclusion of its readme as a term of the license, and for the latter the GPL requires that the license is made known to the recipient (normally by including the license as a text file) and that the source is available (although it isn't with any newgrf I've seen; I rely on the fact that the most common compiler is also a decompiler).

The point is, that these terms of these licenses cannot possibly be maintained by an automatic grf copying mechanism, unless there's some way of giving a list of files that must be copied along with the grf. I'd say that the legal complications stay a whole lot less complicated if the developers just leave grf copying to web sites and forum threads, where licensing terms can be made abundantly clear.

[Quark]

That's a difficult one. The devs won't be responsible for which grfs are distributed, but are they responsible for making it easy for others to distribute them? The Napster scenario.

Are the weapons manufacturers responsible for making it easy to kill peoples?

Is it better for OTTD to assume that the server does not have permission to distribute the grf, or is it better to put the onus on the server to only operate with a grf that they do have permission to distribute?

It's better to make option «automatically distribute grfs» with flagging each grf file that have this permission from author. You can display license text before download if grf file requires it. Just as some updates on windowsupdate site.
Yes, it will require to make installation packages for newgrfs.

[brupje]

It's better to make option «automatically distribute grfs» with flagging each grf file that have this permission from author. You can display license text before download if grf file requires it. Just as some updates on windowsupdate site.
Yes, it will require to make installation packages for newgrfs.

This is an open source project, your suggestion is no limitation what so ever and neither a feature that the end user would want.

[Brianetta]

That's a difficult one. The devs won't be responsible for which grfs are distributed, but are they responsible for making it easy for others to distribute them? The Napster scenario.

Are the weapons manufacturers responsible for making it easy to kill peoples?

I said the Napster scenario, not the gun control argument. Napster facilitated breach of copyright, and despite their protests that they weren't responsible for what was shared, they were deemed responsible in court. Clearly, I'm hoping the devs don't wind up in court, but don't assume that nobody's that vindictive.

Is it better for OTTD to assume that the server does not have permission to distribute the grf, or is it better to put the onus on the server to only operate with a grf that they do have permission to distribute?

It's better to make option «automatically distribute grfs» with flagging each grf file that have this permission from author. You can display license text before download if grf file requires it. Just as some updates on windowsupdate site.
Yes, it will require to make installation packages for newgrfs.

As I said before, it would require a change to every grf out there. Who's going to convince all those newgrf authors (some of whom might be difficult to identify) that they must change their file to allow or disallow automatic copying? The moment you decide that the newgrf specification needs to change is the moment your newgrf library effectively gets truncated.

[Haukinger]

And that's exactly the point - why do we want newgrfs anyway ? Because they are there. They're not easy to learn to create and not easy to learn to understand. With a clean xml-description instead, the learning curve would be much less steep. A newgrf-to-xml-converter is trivial to write once you have a loading mechanism for newgrfs (in ottd) and a specification for xml-newgrfs, so newgrfs could be converted to xml (with the author's consent). Ottd could then use old-school newgrfs and xml-newgrfs, but the xml-specification would be easy to enhance, allowing for new features. And I predict a lot of new trainsets or whatever showing up, if the file-format is easy to learn and to write. BTW, they could contain a flag saying "server is allowed to offer this for automatic download"

[brupje]

And that's exactly the point - why do we want newgrfs anyway ? Because they are there. They're not easy to learn to create and not easy to learn to understand. With a clean xml-description instead, the learning curve would be much less steep. A newgrf-to-xml-converter is trivial to write once you have a loading mechanism for newgrfs (in ottd) and a specification for xml-newgrfs, so newgrfs could be converted to xml (with the author's consent). Ottd could then use old-school newgrfs and xml-newgrfs, but the xml-specification would be easy to enhance, allowing for new features. And I predict a lot of new trainsets or whatever showing up, if the file-format is easy to learn and to write. BTW, they could contain a flag saying "server is allowed to offer this for automatic download"

Yeah that makes it really easy to replace no with yes

I agree that XML is a more portable format.

[DaleStan]

BTW, they could contain a flag saying "server is allowed to offer this for automatic download"

Yeah that makes it really easy to replace no with yes

But it doesn't make it any easier to replace "<This flag does not exist>" to "Yes", and that's the change you're discussing.

I agree that XML is a more portable format.

Why? What makes one plain-text format any more portable than some other format?

And that's exactly the point - why do we want newgrfs anyway ? Because they are there. They're not easy to learn to create and not easy to learn to understand. With a clean xml-description instead

Has it occurred to you that there might be a reason no one has specced a "clean xml description"? You go ahead an write one. In the meantime, Ill stick to my NFO.

I knew that keeping these on hand would eventually be useful:

Code: Select all

<patchman> it seems to me that most of the opposition to NFO in OTTD is either (a) "I didn't design it so it must be crap" or (b) "I have no idea how it works so it must be crap"
<peter1138> though i think DaleStan might be pushing the point too much ;)
<patchman> well, it's a point worth pushing
<patchman> and besides hyperbole and FUD the opposition seems to have made no valid arguments against it

It seems to me like there are really two options here:

1. Write a program that takes your "plain text" spec file and converts it to NFO, then use the already-(mostly-)working NFO support in OTTD and be done. [1]
2. Write a program that takes your "plain text" spec file and converts it to some other, yet-to-be-specified easily computer-readable format. Then start from scratch implementing this format (which for NFO took many, many months and won't be significantly faster no matter how nicely you design your new one), fixing bugs, redoing the specs because you find things you need to do that can't be done, restart coding yet again, fixing the limitations of specs. Deal with the headaches of supporting two incompatible formats internally (unless you want to drop support for all existing graphics). Repeat as necessary.

My point is, why not use a format that works, can do just about everything that needs to be done[2], that there are many people out there who know how it works, and that OTTD mostly supports already? Why bother making all the mistakes that TTDPatch made all over again in a new design?

I'm not saying NFO is perfect. It has warts, and some things aren't designed as cleverly or cleanly as they could've been (because it's a grown design that grew as artist's demands increased). However, it is very simple for doing simple things (you can make a new train with four or five lines of very simple "code". It can get very complex too, but only when complex things need to be done. You cannot remove the complexity without losing its power and sophistication. Any new design will need to be just as complex if it needs to be able to do what NFO can do[3].

If the hex-ness of NFO is the only thing that bothers you, then it's the only thing that needs fixing. This means either improving/fixing GRFMaker, or writing a "plain text"->NFO converter. A variation of the latter would need to be written even if you come up with your own format.

So why not NFO?

[1] Whether the graphics are 8bpp or 32bpp or 256bpp does not matter here. The way sprites are encoded in the file is irrelevant. NFO can work just as well no matter what type of sprites you have.
[2] And can relatively easily be extended to support new things.
[3] For example Locomotion has a very simple format. As a result it can only do simple things. No repainting of wagons after N years to show a new historical livery. No showing of dirt as vehicles get older. No random variations in colours. No seasonal cargo acceptance/production of industries. No sophisticated animation control. No sophisticated station designs. No simple support for trainsets, other than defining a new vehicle ID for each loco and each wagon in each train set. Etc...

A couple of my posts in that thread, though by now rather out-of-date, may also be instructive.

[richk67]

I think the general point is that XML is a more human-readable (and therefore debuggable) format than NFO.

However, I thoroughly agree with Patchman's strategy 1). NFO is there, and works. All that is needed is a more human-friendly interface to NFO.

In theory (because no-one has yet written one), a XML to NFO converter should be relatively straightforward, as a) NFO is well defined, and b) it would be easy to associate a clear XML string with an NFO attribute.

IMO, better yet would be a gui editor of .grf files that effectively internalises all NFO and grfcodec functions.

Back on the main topic - the necessity to do .grfs via an NFO intermediate format is IMO an unnecessary restriction. Both Patch and OTTD should have better .grf support tools.

[Quark]

Why not just define some extensions to grf and simple attach it to end of file?

[DaleStan]

In theory (because no-one has yet written one), a XML to NFO converter should be relatively straightforward, as a) NFO is well defined, and b) it would be easy to associate a clear XML string with an NFO attribute.

I'll grant that the XML <-> NFO conversion is indeed easy for action 0, as long as you restrict <nID> to 1 (which most grf authors do anyway), but, that (by size of the wiki's HTML files) constitutes only about 1/3 of the spec.

The remaining two-thirds contain, among other things, a Turing-complete preprocessor, a command that can arbitrarily change the content of the following pseudo-sprite, and a language that can perform almost any non-looping combination of C's +, -, /, *, %, ^, |, and & operators, as well as the min and max functions, on up to, and beyond, 10K of input data.
That still holds even if we restrict ourselves to the non-advanced action 2s. The math becomes somewhat more restricted, but I still can't find a reasonable construct that can't be encoded. Furthermore, the action 1/2/3 chains are webs, not trees, which XML, to my knowledge, handles poorly.

[brupje]

BTW, they could contain a flag saying "server is allowed to offer this for automatic download"

Yeah that makes it really easy to replace no with yes

But it doesn't make it any easier to replace "<This flag does not exist>" to "Yes", and that's the change you're discussing.

I was trying to make the point that it's useless to have a (copyright) protection of any kind in an open source program. Especially if you are using a XML file, and try to protect a file with a yes/no flag.

t;]I agree that XML is a more portable format.

Why? What makes one plain-text format any more portable than some other format?

because it's specifications are open and widely in use and it's 'friendly' editable with one of the many XML editors.

[Darkvater]

I agree with Dalestan's post above. There is already a GUI for grf, called GRFMaker, which after I think about 2 years of development is still in such a beta fase that it's not publicly available unless by specific request and for the advanced part of the grf spec coders still need to revert to coding by hand because GRFMaker does not support it.

I'm not blaming GRFMaker for this because it is very hard to make a GUI for some of the options Dalestan mentioned so I think that is a pretty fair comparison to writing a (translated) spec in XML.

[Haukinger]

And that's exactly the point - why do we want newgrfs anyway ? Because they are there. They're not easy to learn to create and not easy to learn to understand. With a clean xml-description instead

Has it occurred to you that there might be a reason no one has specced a "clean xml description"? You go ahead an write one. In the meantime, Ill stick to my NFO.

I knew that keeping these on hand would eventually be useful:

Code: Select all

<patchman> it seems to me that most of the opposition to NFO in OTTD is either (a) "I didn't design it so it must be crap" or (b) "I have no idea how it works so it must be crap"
<peter1138> though i think DaleStan might be pushing the point too much ;)
<patchman> well, it's a point worth pushing
<patchman> and besides hyperbole and FUD the opposition seems to have made no valid arguments against it

I vote for b Seriously, NFO a bit hard to learn. Maybe it's easy and effective to use once you've learned it, but that takes a while.
The main argument against I see against xml is claiming that it introduces overhead for ottd to load and parse it. That's undeniably true, but misses the point. Ottd will read the file at startup and then store everything in internal data-structures. Who cares whether startup takes a few second longer (and in a few seconds, tinyXML can parse a trainload of xml-files) ?

Anyway, I believe *any* xml-spec is easier to write and read than nfo for everyone except nfo-experts.
Example:

Code: Select all

<?xml version="1.0" ?>
<OTTDScripting version="1.5" />
<Trainset name="Haukinger's Trainset">
<Engine name="Tiny, electric Engine">
<Motor railtype="electric" power="250kw" />
<Sprites>
<Default name="teE.png" />
</Sprites>
</Engine>
<Engine name="Highpower nuclear Engine" />
<Motor railtype="normal" power="35000kw" />
<Sprites>
<Default name="hpnE.png" />
<Special name="boom.png" function="whenToExplode.lua" />
</Sprites>
</Engine>
<Car name="My passenger Car">
<Cargo type="Passengers" capacity="100" />
<Weight empty="50t" full="58t" />
<Sprites>
<Default name="mpc.png" />
</Sprites>
</Car>
</Trainset>

Now tell me you don't understand what this file tells ottd to do.
And if someone is unhappy because he can't specify that his engine is a transformer that morphs between coal-truck and maglev-engine, he can add a morphing-engine-type to ottd and update the scripting-version to 1.6 and it's done.
Ottd's savegames aren't compatible with ttdpatch, with a reason I assume. Why not extend the data-files too ? And if extending, why not use a plain text format, that everyone can read and understand (xml for example) ? Why write a parser if good parsers exist (timyXML for example) ?
Opposition to NFO in ottd from my point of view is mainly because it makes things more complicated for many (potential trainset coders) because few (ottd coders) don't want to change/extend existing code (include an xml-parser/scripting language).

[Haukinger]

I was trying to make the point that it's useless to have a (copyright) protection of any kind in an open source program. Especially if you are using a XML file, and try to protect a file with a yes/no flag.

Of course there will always be hackers around, but that's what police is for. If a newgrf could say "don't distribute me", it would be an advancement. Not in the sense of enforcing that no copies are distributed but to inform users that the author doesn't want that.

[Rubidium]

Why write a parser if good parsers exist (timyXML for example) ?

Well, maybe because you need a second parser to parse the DOM to translate it into useable in-game information?

Secondly, how would you incorporate sprites? Adding binary data would defy the reason why you choose for XML as it will (probly) not work right in a text editor. The nice thing of the current (New)GRF format is that all sprites can be read directly from the files into the memory when needed. No parsing whatsoever, no opening files (as the GRF is already opened). To avoid opening lots of files during the game with XML, you have to load all of them into memory (and keeping them there). This will use quite some memory when it is totally unneccessary.