Is there still a plan to remove unnecessary chunks from the vanilla server's map?
Quote from: Thorin on February 14, 2013, 09:33:16 AM
Is there still a plan to remove unnecessary chunks from the vanilla server's map?
There are still people that play on the vanilla server? ;)
Quote from: Thorin on February 14, 2013, 09:33:16 AM
Is there still a plan to remove unnecessary chunks from the vanilla server's map?
I've slowly been working on the tool. I actually had it properly loading and able to scan the map, but when starting on support for writing the map back out, the existing loading/handling code was annoying me, so I started on a new class to handle that in a much cleaner less copy/paste prone way. That work is close to written, will just need to spend some time fixing bugs, then I can do the scanning code.
Quote from: Lazybones on February 14, 2013, 09:37:42 AM
Quote from: Thorin on February 14, 2013, 09:33:16 AM
Is there still a plan to remove unnecessary chunks from the vanilla server's map?
There are still people that play on the vanilla server? ;)
Yeah, I tried Feed The Beast and was annoyed at and overwhelmed with all of the new options. I really don't feel like learning seventeen new tech trees just to build something, I'm fine with using vanilla MC building blocks. However, there is a need to trim the map so that new blocks coming out in the new versions of Minecraft can be found. And also, if there were less chunks loaded due to people setting and exploring for days, the Overviewer map would be a lot more zoomed in at first glance.
Maybe I'll finally get some good crafting and mining in this weekend, although there's yet another tournament to attend so maybe not...
Tom: is it in a workable state? Or will it be quite a while yet?
Quote from: Thorin on February 14, 2013, 09:55:22 AM
I really don't feel like learning seventeen new tech trees just to build something
Start with one, like everyone else :P
Quote from: Thorin on February 14, 2013, 09:55:22 AM
Tom: is it in a workable state? Or will it be quite a while yet?
It's not working right now, but as I said in my last post, the rework is almost done, then I can write the scanner/deleter. The hard work is getting the loading and saving code /right/. I'll need to do some thorough testing to make sure it saves stuff in a way minecraft likes. I got a bunch done earlier this week. Assuming I have as much free time later this week as I had early this week, it won't be too much longer.
Right now (at work) I'm just waiting on another developer to finish some remote/api work, so I can get the app using the new api calls.
Quote from: Thorin on February 14, 2013, 09:55:22 AM
Yeah, I tried Feed The Beast and was annoyed at and overwhelmed with all of the new options. I really don't feel like learning seventeen new tech trees just to build something, I'm fine with using vanilla MC building blocks.
I know, I was joking hence the ;)..
As far as FTB goes picking one mod that interests you, then picking one item to build and finding out what resources are needed to make it is the way to go. Getting into mixing mods comes later, and the only thing you need to get over is ignoring resources you mine but have no use for.... I started storing all sorts of minerals and metals that I had no use for then came back to them when I found a use for them.
The best mod to get started with is probably BuildCraft as it is fairly easy to get the basics started and has a great return on investment for the effort you put in
Other mods like factorization are just pure pain haha
Quote from: Mr. Analog on February 14, 2013, 11:44:53 AM
The best mod to get started with is probably BuildCraft as it is fairly easy to get the basics started and has a great return on investment for the effort you put in
Other mods like factorization are just pure pain haha
Let's move to the pure FTB modpack (rather than DW20), then we can start playing with GregTech.
As long as it doesn't break my existing stuff :)
Oh yes, I would like to say that I am looking forward to the map cleanup, including the Nether
Quote from: Mr. Analog on February 14, 2013, 03:50:34 PM
As long as it doesn't break my existing stuff :)
Oh yes, I would like to say that I am looking forward to the map cleanup, including the Nether
Yeah, I wasn't originally planning on cleaning the nether, but it's kind of silly not to now with all the additions.
As for breakage... if(block.placedBy("Mr_Analog")) { block.delete(); }
;D
Somebody gonna get a hurt
Quote from: Mr. Analog on February 14, 2013, 03:50:34 PM
As long as it doesn't break my existing stuff :)
See, that's how I feel about my stuff on the vanilla server, including what I've built in the nether. Don't want it to all just suddenly disappear. Although if Tom can get me a list of all the places where there are three or more obsidian blocks on top of each other and above level 30 on the overworld, I can at least go back and put everyone's portals back in (and make a nice grid-based tunnel system in the nether while I'm at it). But I do need to know where those portals are in the overworld, and three obsidian blocks on top of each other above level 30 means portal...
Quote from: Thorin on February 14, 2013, 08:58:55 PM
Quote from: Mr. Analog on February 14, 2013, 03:50:34 PM
As long as it doesn't break my existing stuff :)
See, that's how I feel about my stuff on the vanilla server, including what I've built in the nether. Don't want it to all just suddenly disappear. Although if Tom can get me a list of all the places where there are three or more obsidian blocks on top of each other and above level 30 on the overworld, I can at least go back and put everyone's portals back in (and make a nice grid-based tunnel system in the nether while I'm at it). But I do need to know where those portals are in the overworld, and three obsidian blocks on top of each other above level 30 means portal...
The idea of doing this is so that stuff isn't lost, otherwise I'd have just said, NEW MAP TIME! But the three obsidian blocks on top of each other is a good test, just in case theres a gate in the middle of nowhere with no "odd" placed blocks (stuff that doesn't normally generate automatically).
Oh, there are portals in the middle of nowhere and in the middle of the air... So are you going to hand-mark the map or does this tool automate looking for built-up areas? We could always just go put a netherrack block in every chunk we don't want deleted in the overworld...
Also, NEW MAP TIME would've sucked balls! I want to keep all the things I've built, and I'm sure everyone else does, too.
Quote from: Thorin on February 14, 2013, 09:29:45 PM
Oh, there are portals in the middle of nowhere and in the middle of the air... So are you going to hand-mark the map or does this tool automate looking for built-up areas? We could always just go put a netherrack block in every chunk we don't want deleted in the overworld...
Hm, thats not a half bad idea. But then we'd end up with a piece of netherrack in every 16x16 square area, and thats just evil.
The tool will scan for blocks/entities or maybe combinations of blocks that the map generator won't do. Like glass, half slabs, glowstone lamps, jackolanterns, noob towers ;), fences/gates, redstone of any kind, dispensers (well, they can be generated by the game, but they are quite rare), double chests, levers, buttons, trip wire, pistons, just about any "tile entitiy" really, or three obsidian in a stack.
If you guys can think of any clever checks I could make that would be very accurate, and not be too time intensive to check, please let me know.
Quote from: Thorin on February 14, 2013, 09:29:45 PM
Also, NEW MAP TIME would've sucked balls! I want to keep all the things I've built, and I'm sure everyone else does, too.
Even I don't want to lose my stuff, even though I have very little stuff on the vanilla server.
Well the presence of torches is in vanilla is rather common, they can be generated but mostly they are a sign of life.
Rail tracks are also another one.
Quote from: Lazybones on February 14, 2013, 10:05:44 PM
Well the presence of torches is in vanilla is rather common, they can be generated but mostly they are a sign of life.
Rail tracks are also another one.
I thought about that, but that will pull in abandoned mines, and meh, maybe we want to regen those. Also why I don't want to check for cobble, brick, or stairs, even though they are mostly a sign of life, they also gen fairly commonly.
I don't mind having a netherrack block in every chunk we want to keep, I'd be happy to go back and remove them all afterwards.
Quote from: Thorin on February 15, 2013, 12:16:25 AM
I don't mind having a netherrack block in every chunk we want to keep, I'd be happy to go back and remove them all afterwards.
I'm probably going to try a bunch of things. Also I was planning on keeping X chunks around a given detected junk as well. So like say it found something in a chunk, it would probably also pull in the 8 chunks immediately around it (3x3 square). Or maybe even go for a diameter of 2 or 3. I'll try a bunch of things.
Or we could remove the portals that aren't used
I like where you are going with the changes though Tom sounds like the right track
Then we would need to check with people if they use their portals. Sure, there's Tele-Portals now that teleport us places, but I find myself still using nether portals at times. And I have no idea what Aethun or Pants or KDevil or Lennylen or SnowieOwl or anyone else uses, since I don't see them on that often to talk to them (some because of wildly different time zones, others because they don't log on very much).
Quote from: Thorin on February 15, 2013, 09:53:34 AM
Then we would need to check with people if they use their portals. Sure, there's Tele-Portals now that teleport us places, but I find myself still using nether portals at times. And I have no idea what Aethun or Pants or KDevil or Lennylen or SnowieOwl or anyone else uses, since I don't see them on that often to talk to them (some because of wildly different time zones, others because they don't log on very much).
It doesn't really matter if we leave in a few unused portals, even if they are broken. Actually I can check for the actual portal block rather than obsidian I think. Then it would only pull in active portals.
So the loading/saving work is done :o
Managed to find the desire to work on it a little here and there. Spent some time fighting with bad documentation, so it wasn't /all/ my fault ;D
Anyhow, I thought you guys might be interested in some pointless numbers:
total regions: 997
total chunks: 431680
total blocks: 7964909568
0: Air: 1049521976 13.18%
1: Stone: 4567287005 57.34%
2: Grass: 27636575 0.35%
3: Dirt: 553631652 6.95%
4: Cobblestone: 1083587 0.01%
5: Wooden Plank: 5366050 0.07%
Wooden Plank: 5314316 99.04%
Wooden Plank (Pine): 5763 0.11%
Wooden Plank (Birch): 43222 0.81%
Wooden Plank (Jungle): 2749 0.05%
6: Sapling: 147 0.00%
Sapling: 108 73.47%
Sapling (Spruce): 24 16.33%
Sapling (Birch): 15 10.20%
Sapling (Jungle): 0 0.00%
7: Adminium: 331409395 4.16%
8: Water: 83993 0.00%
9: Stationary Water: 948047658 11.90%
10: Lava: 21551853 0.27%
11: Stationary Lava: 26415128 0.33%
12: Sand: 116402192 1.46%
13: Gravel: 147145204 1.85%
14: Gold Ore: 3468715 0.04%
15: Iron ore: 30986878 0.39%
16: Coal Ore: 55423209 0.70%
17: Wood: 2905116 0.04%
Wood: 835896 28.77%
Wood (Pine): 467158 16.08%
Wood (Birch): 144866 4.99%
Wood (Jungle): 1457196 50.16%
18: Leaves: 24944777 0.31%
Leaves: 14972721 60.02%
Leaves (Pine): 3465762 13.89%
Leaves (Birch): 1237805 4.96%
Leaves (Jungle): 5268489 21.12%
19: Sponge: 0 0.00%
20: Glass: 33575 0.00%
21: Lapis Lazuli Ore: 1444032 0.02%
22: Lapis Lazuli Block: 42 0.00%
23: Dispenser: 62 0.00%
24: Sandstone: 10240185 0.13%
Sandstone: 10224928 99.85%
Sandstone (Decorative): 0 0.00%
Sandstone (Smooth): 15257 0.15%
25: Note Block: 4 0.00%
26: Bed: 380 0.00%
27: Powered Rail: 960 0.00%
28: Detector Rail: 9 0.00%
29: Sticky Piston: 165 0.00%
30: Cobweb: 996278 0.01%
31: Grass: 2330654 0.03%
Tall Grass (Shrub): 2188979 93.92%
Tall Grass (Grass): 0 0.00%
Tall Grass (Fern): 141675 6.08%
32: Dead Bush: 10714 0.00%
33: Piston: 69 0.00%
34: Piston Extension: 87 0.00%
35: Wool: 14627 0.00%
Wool: 4071 27.83%
Orange Wool: 581 3.97%
Magenta Wool: 216 1.48%
Light Blue Wool: 134 0.92%
Yellow Wool: 445 3.04%
Light Green Wool: 0 0.00%
Pink Wool: 23 0.16%
Gray Wool: 599 4.10%
Light Gray Wool: 2 0.01%
Cyan Wool: 5 0.03%
Purple Wool: 235 1.61%
Blue Wool: 1136 7.77%
Brown Wool: 635 4.34%
Dark Green Wool: 1232 8.42%
Red Wool: 2005 13.71%
Black Wool: 3308 22.62%
36: Block Moved By Piston: 0 0.00%
37: Yellow Flower: 91749 0.00%
38: Red Flower: 22967 0.00%
39: Brown Mushroom: 44957 0.00%
40: Red Mushroom: 20775 0.00%
41: Gold Block: 99 0.00%
42: Iron Block: 84 0.00%
43: Double Slabs: 6777 0.00%
44: Stone Slab: 24171 0.00%
Stone Slab: 12673 52.43%
Sandstone Slab: 0 0.00%
Wooden Slab: 9777 40.45%
Cobblestone Slab: 0 0.00%
Brick Slab: 1721 7.12%
Stone Brick Slab: 0 0.00%
Alternative Stone Slab: 0 0.00%
45: Brick: 1200 0.00%
46: TNT: 54 0.00%
47: Bookshelf: 2062 0.00%
48: Moss Stone: 388778 0.00%
49: Obsidian: 745591 0.01%
50: Torch: 149366 0.00%
51: Fire: 17743 0.00%
52: Monster Spawner: 18694 0.00%
53: Wooden Stairs: 6795 0.00%
54: Chest: 33681 0.00%
55: Redstone Wire: 1723 0.00%
56: Diamond Ore: 1294087 0.02%
57: Diamond Block: 1 0.00%
58: Workbench: 616 0.00%
59: Wheat: 7694 0.00%
60: Soil: 9662 0.00%
61: Furnace: 842 0.00%
62: Burning Furnace: 8 0.00%
63: Signpost: 80 0.00%
64: Wooden Door: 3042 0.00%
65: Ladder: 5235 0.00%
66: Minecart Track: 1032313 0.01%
67: Cobblestone Stairs: 4347 0.00%
68: Wall Sign: 958 0.00%
69: Lever: 93 0.00%
70: Stone Pressure Plate: 252 0.00%
71: Iron Door: 246 0.00%
72: Wooden Pressure Plate: 358 0.00%
73: Redstone Ore: 10448463 0.13%
74: Glowing Redstone Ore: 148 0.00%
75: Redstone Torch (Off): 51 0.00%
76: Redstone Torch (On): 1156 0.00%
77: Stone Button: 271 0.00%
78: Snow: 8357370 0.10%
79: Ice: 1626223 0.02%
80: Snow Block: 588 0.00%
81: Cactus: 15999 0.00%
82: Clay: 3318515 0.04%
83: Sugar Cane: 16021 0.00%
84: Jukebox: 2 0.00%
85: Fence: 3853109 0.05%
86: Pumpkin: 1558 0.00%
87: Netherrack: 3541 0.00%
88: Soul Sand: 81 0.00%
89: Glowstone: 1140 0.00%
90: Portal: 342 0.00%
91: Jack-O-Lantern: 254 0.00%
92: Cake: 15 0.00%
93: Repeater (off): 237 0.00%
94: Repeater (on): 75 0.00%
95: Locked Chest: 0 0.00%
96: Trapdoor: 170 0.00%
97: Monster Egg: 6396 0.00%
98: Stone Brick: 130378 0.00%
Stone Brick: 104151 79.88%
Cracked Stone Brick: 15749 12.08%
Missy Stone Brick: 10464 8.03%
Chiseled Stone Brick: 14 0.01%
99: Huge brown mushroom: 10475 0.00%
100: Huge Red Mushroom: 11203 0.00%
101: Iron Bars: 1703 0.00%
102: Glass Pane: 27722 0.00%
103: Melon: 318 0.00%
104: Pumpkin Stem: 234 0.00%
105: Melon Stem: 361 0.00%
106: Vines: 4490685 0.06%
107: Fence Gate: 214 0.00%
108: Brick Stairs: 290 0.00%
109: Stone Brick Stairs: 5183 0.00%
110: Mycelium: 179870 0.00%
111: Lily Pad: 42841 0.00%
112: Nether Brick: 361 0.00%
113: Nether Brick Fence: 111 0.00%
114: Nether Brick Stairs: 68 0.00%
115: Nether Wart: 81 0.00%
116: Enchantment Table: 17 0.00%
117: Brewing Stand: 5 0.00%
118: Cauldron: 28 0.00%
119: End Portal: 0 0.00%
120: End Portal Block: 24 0.00%
121: End Stone: 0 0.00%
122: Dragon Egg: 0 0.00%
123: Redstone Lamp (off): 39 0.00%
124: Redstone Lamp (on): 118 0.00%
125: Wooden Double Slab: 823 0.00%
126: Wooden Slab: 2759 0.00%
Oak-Wood Slab: 1460 52.92%
Spruce-Wood Slab: 202 7.32%
Birch-Wood Slab: 1097 39.76%
Jungle-Wood Slab: 0 0.00%
127: Cocoa Plant: 3253 0.00%
128: Sandstone Stairs: 1044 0.00%
129: Emerald Ore: 25638 0.00%
130: Ender Chest: 3 0.00%
131: Tripwire Hook: 14 0.00%
132: Tripwire: 17 0.00%
133: Block of Emerald: 0 0.00%
134: Spruce Wood Stairs: 776 0.00%
135: Birch Wood Stairs: 540 0.00%
136: Jungle Wood Stairs: 0 0.00%
137: Command Block: 30 0.00%
138: Beacon: 0 0.00%
139: Cobblestone Wall: 186 0.00%
Cobblestone Wall: 186 100.00%
Mossy Cobblestone Wall: 0 0.00%
140: Flower Pot: 22 0.00%
141: Carrots: 193 0.00%
142: Potatoes: 149 0.00%
143: Wooden Button: 2 0.00%
144: Mob Head: 4 0.00%
145: Anvil: 8 0.00%
That's the entire overworld as of about an hour before this post.
You read that right, > 430000 chunks, almost 8 BILLION blocks. Of course a large portion of them are air and stone. :D most of the air blocks won't even be counted due to the way chunks are saved in layers (to skip large empty sections).
Earlier today I managed to get it loading region files, that it previously saved, after having read them in. So at the very least it can read in existing files, and read in files it itself saved. Next big test will be to do some fiddling, and then try loading up the changed map in minecraft itself.
380 beds?!
Oh, also billions of blocks :-)
So how are you determining which chunks to keep?
Quote from: Thorin on March 04, 2013, 04:55:07 AM
380 beds?!
A bed is technically two blocks, so its more like 190 :)
Quote from: Thorin on March 04, 2013, 04:55:07 AM
Oh, also billions of blocks :-)
Almost 8 of them.
Quote from: Thorin on March 04, 2013, 04:55:07 AM
So how are you determining which chunks to keep?
Not sure yet.
I'm actually planning on making a simple map viewer so I can tell how things look in real time.
Right now, I'm playing with making a dumb nbt editor in Qt. See what falls out of my api ;)
Man this is great, you really get a sense of what's out there, especially how much unused space there is
Also I know where that one block of diamond is haha
Thanks for continuing to work on this. Someday we'll be able to clean our map up :)
One block of diamond, eh? I think I know where the blocks of lapis lazuli are, as well.
And over 57% of the map is stone, even including air and water blocks! So we'll be building stone everything for quite a while yet.
And there's still nearly 1.3 million diamond ores to find, with Fortune III on your pick that's about 5 million diamonds!
Unfortunately most of them are under the ocean haha
Here's a little something I whipped up as a test, and just so I have a decent cross platform nbt editor...
(http://i.imgur.com/3z5fEwJ.png)
Took a few hours, but I've gotten rusty with Qt :( especially the model api it has.
oh, I should probably tell you exactly what that is... As I said, its an nbt editor, or more specifically a minecraft .dat editor. The file loaded in that picture is sevalecan's player.dat file :D
Nice little tool, you're having fun learning the MC data structure?
Quote from: Thorin on March 04, 2013, 02:39:33 PM
Nice little tool, you're having fun learning the MC data structure?
I don't know that I'd call it fun. The way some things are stored is just stupid, but then again, that stupidity is what allows for some of minecraft's flexibility when they make map changes. Did you know that they didn't increase the size of the data type that stores the block ids? Instead they added another field to the crazy NBT chunk structure that stores 4 more bits per block that you shift left, and then add to the normal block id. It's a weee bit crazy. And a bit of a pain.
Hmm, I'm not even checking for that data at the moment, though I doubt anything uses those added block ids. at least the regular blocks don't. I think though that items (like potions) may, and mods definitely do. Something I need to remember to add support for. It was annoying enough to get it parsing the "damage" values for items. For many blocks, the 4 bits for the damage values on blocks/items is shared between a counter and "additional block type" flags.
That qnbt tool is mainly just so I have more than one real app using the libnbt/libmcmap code, and something to muck around with when I need to massage some level or player data. One thing I want to add is a "inventory" mode that makes it a lot easier to mess with a player's inventory, right now you need to have the item id's and damage values memorized. Oh, and saving support. I just added "editing" support to that tree view, but there's no save function wired up to the ui yet.
Took somewhere around 5 hours to make, including time spent fixing/changing code in the libnbt code to make it more friendly to real use cases.
So with 1.5 out and us waiting patiently for a Bukkit update, how's your map cleanup tool coming along?
I honestly have no problem switching to creative mode, flying around, and placing, say, a netherrack block in the top left of every chunk that needs saving. Then you could simply delete all the chunks that don't have netherrack in them, post the map for people to look over, and be done with it all. We could remove the netherrack, or even just leave it there.
As for the nether, I don't want it blown away, but I want quartz and easy access to glowstone, so... Mr. A is starting to win me over to the dark side on that one :) If you could supply a list of coordinates where there are three obsidian blocks stacked above each other, or even better three nether portal blocks (although that would only find the active ones), I'd be happy to go rebuild the nether portals in the nether and their connecting corridors (likely in a standard grid pattern every X blocks).
Aw man, it would be cool if we could start playing with the new stuff!
It's going well. It loads and saves NBT files. Have to test it with loading them back into minecraft. Then to write some scanning code.
Quote from: Tom on March 14, 2013, 03:40:46 PM
It's going well. It loads and saves NBT files. Have to test it with loading them back into minecraft. Then to write some scanning code.
Nice! Looking forward to testing the map out :)
I think we have to wait with the new stuff, since we're using Bukkit for our "vanilla" server and thus can't upgrade until there's a stable Bukkit upgrade.
Tom, any news on the Bukkit front?
Quote from: Thorin on March 14, 2013, 04:33:10 PM
I think we have to wait with the new stuff, since we're using Bukkit for our "vanilla" server and thus can't upgrade until there's a stable Bukkit upgrade.
Tom, any news on the Bukkit front?
I didn't look till just now, and not very hard. 1.5 is a pretty big update, so we might be waiting for a few weeks or more. Unless they kept up with the snapshots, or got pre-releases from mojang. But yeah, may be a while going on history.
So... how is this progressing (assuming Bukkit is updated soon).
I mean, I guess I could just spawn a ton of quartz in hehe
Tom, is this tool something we can help you with? Mr. A has mentioned he's ready to just dump the existing map and start over so we can get at all the new materials that are supposed to be available. I don't want to lose all my builds (like that giant airplane!), so if I can help you trim down the existing map, that'd be really nice.
Just a crazy thought I guess, like make the map downloadable (like the old Tekkit one) and then just refresh the world?
I'd love to get my hands new ores and stuff (like emerald!), I plan on starting some new builds I guess anyway but I want to wait until the new ores are there (I have big plans for Quartz when it's available)
Like quartz decorative blocks look like doric columns which are particularly well suited to Neo-Classical architecture y'know, the kind used in the flying cities of 1912 lol
It shouldn't take too much work. I've just been sidetracked lately. I can probably have something working without too much trouble.
Awesome!
Well, you're doing this for free for us so let us know if you need any help with anything. We're all pretty tech- and code-savvy. At the same time, we're eagerly awaiting a cleaned-up map so we can go mining for the newer resources!
Also, have we talked any further about blowing away the nether?
Quote from: Thorin on April 03, 2013, 01:25:17 PM
Well, you're doing this for free for us so let us know if you need any help with anything. We're all pretty tech- and code-savvy. At the same time, we're eagerly awaiting a cleaned-up map so we can go mining for the newer resources!
Also, have we talked any further about blowing away the nether?
I don't really see a point in not just running the same code on the nether as the overworld. Get rid of useless chunks to generate more goodies :D
But I don't have anything against just deleting the nether. it'd make re-homing the gates on that side annoying though. So just trimming the nether is probably easier.
Have to say though, my rss project has taken top priority in my personal project time. But I'll push up the cleanup for a bit. try and finish it up.
If you want to run the nether through that same tool, that'd be cool. Hopefully we'd get a bunch of chunks freed up so that we'd be able to find nether quartz.