vanilla map cleanup

[Tom]

Oh, there are portals in the middle of nowhere and in the middle of the air...  So are you going to hand-mark the map or does this tool automate looking for built-up areas?  We could always just go put a netherrack block in every chunk we don't want deleted in the overworld...

Hm, thats not a half bad idea. But then we'd end up with a piece of netherrack in every 16x16 square area, and thats just evil.

The tool will scan for blocks/entities or maybe combinations of blocks that the map generator won't do. Like glass, half slabs, glowstone lamps, jackolanterns, noob towers , fences/gates, redstone of any kind, dispensers (well, they can be generated by the game, but they are quite rare), double chests, levers, buttons, trip wire, pistons, just about any "tile entitiy" really, or three obsidian in a stack.

If you guys can think of any clever checks I could make that would be very accurate, and not be too time intensive to check, please let me know.

Also, NEW MAP TIME would've sucked balls!  I want to keep all the things I've built, and I'm sure everyone else does, too.

Even I don't want to lose my stuff, even though I have very little stuff on the vanilla server.

[Lazybones]

Well the presence of torches is in vanilla is rather common, they can be generated but mostly they are a sign of life.

Rail tracks are also another one.

[Tom]

Well the presence of torches is in vanilla is rather common, they can be generated but mostly they are a sign of life.

Rail tracks are also another one.

I thought about that, but that will pull in abandoned mines, and meh, maybe we want to regen those. Also why I don't want to check for cobble, brick, or stairs, even though they are mostly a sign of life, they also gen fairly commonly.

[Thorin]

I don't mind having a netherrack block in every chunk we want to keep, I'd be happy to go back and remove them all afterwards.

[Tom]

I don't mind having a netherrack block in every chunk we want to keep, I'd be happy to go back and remove them all afterwards.

I'm probably going to try a bunch of things. Also I was planning on keeping X chunks around a given detected junk as well. So like say it found something in a chunk, it would probably also pull in the 8 chunks immediately around it (3x3 square). Or maybe even go for a diameter of 2 or 3. I'll try a bunch of things.

[Mr. Analog]

Or we could remove the portals that aren't used

I like where you are going with the changes though Tom sounds like the right track

[Thorin]

Then we would need to check with people if they use their portals.  Sure, there's Tele-Portals now that teleport us places, but I find myself still using nether portals at times.  And I have no idea what Aethun or Pants or KDevil or Lennylen or SnowieOwl or anyone else uses, since I don't see them on that often to talk to them (some because of wildly different time zones, others because they don't log on very much).

[Tom]

Then we would need to check with people if they use their portals.  Sure, there's Tele-Portals now that teleport us places, but I find myself still using nether portals at times.  And I have no idea what Aethun or Pants or KDevil or Lennylen or SnowieOwl or anyone else uses, since I don't see them on that often to talk to them (some because of wildly different time zones, others because they don't log on very much).

It doesn't really matter if we leave in a few unused portals, even if they are broken. Actually I can check for the actual portal block rather than obsidian I think. Then it would only pull in active portals.

[Tom]

So the loading/saving work is done

Managed to find the desire to work on it a little here and there. Spent some time fighting with bad documentation, so it wasn't /all/ my fault

Anyhow, I thought you guys might be interested in some pointless numbers:

total regions: 997
total chunks: 431680
total blocks: 7964909568
0: Air: 1049521976 13.18%
1: Stone: 4567287005 57.34%
2: Grass: 27636575 0.35%
3: Dirt: 553631652 6.95%
4: Cobblestone: 1083587 0.01%
5: Wooden Plank: 5366050 0.07%
  Wooden Plank: 5314316 99.04%
  Wooden Plank (Pine): 5763 0.11%
  Wooden Plank (Birch): 43222 0.81%
  Wooden Plank (Jungle): 2749 0.05%
6: Sapling: 147 0.00%
  Sapling: 108 73.47%
  Sapling (Spruce): 24 16.33%
  Sapling (Birch): 15 10.20%
  Sapling (Jungle): 0 0.00%
7: Adminium: 331409395 4.16%
8: Water: 83993 0.00%
9: Stationary Water: 948047658 11.90%
10: Lava: 21551853 0.27%
11: Stationary Lava: 26415128 0.33%
12: Sand: 116402192 1.46%
13: Gravel: 147145204 1.85%
14: Gold Ore: 3468715 0.04%
15: Iron ore: 30986878 0.39%
16: Coal Ore: 55423209 0.70%
17: Wood: 2905116 0.04%
  Wood: 835896 28.77%
  Wood (Pine): 467158 16.08%
  Wood (Birch): 144866 4.99%
  Wood (Jungle): 1457196 50.16%
18: Leaves: 24944777 0.31%
  Leaves: 14972721 60.02%
  Leaves (Pine): 3465762 13.89%
  Leaves (Birch): 1237805 4.96%
  Leaves (Jungle): 5268489 21.12%
19: Sponge: 0 0.00%
20: Glass: 33575 0.00%
21: Lapis Lazuli Ore: 1444032 0.02%
22: Lapis Lazuli Block: 42 0.00%
23: Dispenser: 62 0.00%
24: Sandstone: 10240185 0.13%
  Sandstone: 10224928 99.85%
  Sandstone (Decorative): 0 0.00%
  Sandstone (Smooth): 15257 0.15%
25: Note Block: 4 0.00%
26: Bed: 380 0.00%
27: Powered Rail: 960 0.00%
28: Detector Rail: 9 0.00%
29: Sticky Piston: 165 0.00%
30: Cobweb: 996278 0.01%
31: Grass: 2330654 0.03%
  Tall Grass (Shrub): 2188979 93.92%
  Tall Grass (Grass): 0 0.00%
  Tall Grass (Fern): 141675 6.08%
32: Dead Bush: 10714 0.00%
33: Piston: 69 0.00%
34: Piston Extension: 87 0.00%
35: Wool: 14627 0.00%
  Wool: 4071 27.83%
  Orange Wool: 581 3.97%
  Magenta Wool: 216 1.48%
  Light Blue Wool: 134 0.92%
  Yellow Wool: 445 3.04%
  Light Green Wool: 0 0.00%
  Pink Wool: 23 0.16%
  Gray Wool: 599 4.10%
  Light Gray Wool: 2 0.01%
  Cyan Wool: 5 0.03%
  Purple Wool: 235 1.61%
  Blue Wool: 1136 7.77%
  Brown Wool: 635 4.34%
  Dark Green Wool: 1232 8.42%
  Red Wool: 2005 13.71%
  Black Wool: 3308 22.62%
36: Block Moved By Piston: 0 0.00%
37: Yellow Flower: 91749 0.00%
38: Red Flower: 22967 0.00%
39: Brown Mushroom: 44957 0.00%
40: Red Mushroom: 20775 0.00%
41: Gold Block: 99 0.00%
42: Iron Block: 84 0.00%
43: Double Slabs: 6777 0.00%
44: Stone Slab: 24171 0.00%
  Stone Slab: 12673 52.43%
  Sandstone Slab: 0 0.00%
  Wooden Slab: 9777 40.45%
  Cobblestone Slab: 0 0.00%
  Brick Slab: 1721 7.12%
  Stone Brick Slab: 0 0.00%
  Alternative Stone Slab: 0 0.00%
45: Brick: 1200 0.00%
46: TNT: 54 0.00%
47: Bookshelf: 2062 0.00%
48: Moss Stone: 388778 0.00%
49: Obsidian: 745591 0.01%
50: Torch: 149366 0.00%
51: Fire: 17743 0.00%
52: Monster Spawner: 18694 0.00%
53: Wooden Stairs: 6795 0.00%
54: Chest: 33681 0.00%
55: Redstone Wire: 1723 0.00%
56: Diamond Ore: 1294087 0.02%
57: Diamond Block: 1 0.00%
58: Workbench: 616 0.00%
59: Wheat: 7694 0.00%
60: Soil: 9662 0.00%
61: Furnace: 842 0.00%
62: Burning Furnace: 8 0.00%
63: Signpost: 80 0.00%
64: Wooden Door: 3042 0.00%
65: Ladder: 5235 0.00%
66: Minecart Track: 1032313 0.01%
67: Cobblestone Stairs: 4347 0.00%
68: Wall Sign: 958 0.00%
69: Lever: 93 0.00%
70: Stone Pressure Plate: 252 0.00%
71: Iron Door: 246 0.00%
72: Wooden Pressure Plate: 358 0.00%
73: Redstone Ore: 10448463 0.13%
74: Glowing Redstone Ore: 148 0.00%
75: Redstone Torch (Off): 51 0.00%
76: Redstone Torch (On): 1156 0.00%
77: Stone Button: 271 0.00%
78: Snow: 8357370 0.10%
79: Ice: 1626223 0.02%
80: Snow Block: 588 0.00%
81: Cactus: 15999 0.00%
82: Clay: 3318515 0.04%
83: Sugar Cane: 16021 0.00%
84: Jukebox: 2 0.00%
85: Fence: 3853109 0.05%
86: Pumpkin: 1558 0.00%
87: Netherrack: 3541 0.00%
88: Soul Sand: 81 0.00%
89: Glowstone: 1140 0.00%
90: Portal: 342 0.00%
91: Jack-O-Lantern: 254 0.00%
92: Cake: 15 0.00%
93: Repeater (off): 237 0.00%
94: Repeater (on): 75 0.00%
95: Locked Chest: 0 0.00%
96: Trapdoor: 170 0.00%
97: Monster Egg: 6396 0.00%
98: Stone Brick: 130378 0.00%
  Stone Brick: 104151 79.88%
  Cracked Stone Brick: 15749 12.08%
  Missy Stone Brick: 10464 8.03%
  Chiseled Stone Brick: 14 0.01%
99: Huge brown mushroom: 10475 0.00%
100: Huge Red Mushroom: 11203 0.00%
101: Iron Bars: 1703 0.00%
102: Glass Pane: 27722 0.00%
103: Melon: 318 0.00%
104: Pumpkin Stem: 234 0.00%
105: Melon Stem: 361 0.00%
106: Vines: 4490685 0.06%
107: Fence Gate: 214 0.00%
108: Brick Stairs: 290 0.00%
109: Stone Brick Stairs: 5183 0.00%
110: Mycelium: 179870 0.00%
111: Lily Pad: 42841 0.00%
112: Nether Brick: 361 0.00%
113: Nether Brick Fence: 111 0.00%
114: Nether Brick Stairs: 68 0.00%
115: Nether Wart: 81 0.00%
116: Enchantment Table: 17 0.00%
117: Brewing Stand: 5 0.00%
118: Cauldron: 28 0.00%
119: End Portal: 0 0.00%
120: End Portal Block: 24 0.00%
121: End Stone: 0 0.00%
122: Dragon Egg: 0 0.00%
123: Redstone Lamp (off): 39 0.00%
124: Redstone Lamp (on): 118 0.00%
125: Wooden Double Slab: 823 0.00%
126: Wooden Slab: 2759 0.00%
  Oak-Wood Slab: 1460 52.92%
  Spruce-Wood Slab: 202 7.32%
  Birch-Wood Slab: 1097 39.76%
  Jungle-Wood Slab: 0 0.00%
127: Cocoa Plant: 3253 0.00%
128: Sandstone Stairs: 1044 0.00%
129: Emerald Ore: 25638 0.00%
130: Ender Chest: 3 0.00%
131: Tripwire Hook: 14 0.00%
132: Tripwire: 17 0.00%
133: Block of Emerald: 0 0.00%
134: Spruce Wood Stairs: 776 0.00%
135: Birch Wood Stairs: 540 0.00%
136: Jungle Wood Stairs: 0 0.00%
137: Command Block: 30 0.00%
138: Beacon: 0 0.00%
139: Cobblestone Wall: 186 0.00%
  Cobblestone Wall: 186 100.00%
  Mossy Cobblestone Wall: 0 0.00%
140: Flower Pot: 22 0.00%
141: Carrots: 193 0.00%
142: Potatoes: 149 0.00%
143: Wooden Button: 2 0.00%
144: Mob Head: 4 0.00%
145: Anvil: 8 0.00%

That's the entire overworld as of about an hour before this post.

You read that right, > 430000 chunks, almost 8 BILLION blocks. Of course a large portion of them are air and stone. most of the air blocks won't even be counted due to the way chunks are saved in layers (to skip large empty sections).

Earlier today I managed to get it loading region files, that it previously saved, after having read them in. So at the very least it can read in existing files, and read in files it itself saved. Next big test will be to do some fiddling, and then try loading up the changed map in minecraft itself.

[Thorin]

380 beds?!

Oh, also billions of blocks :-)

So how are you determining which chunks to keep?

[Tom]

380 beds?!

A bed is technically two blocks, so its more like 190

Oh, also billions of blocks :-)

Almost 8 of them.

So how are you determining which chunks to keep?

Not sure yet.

I'm actually planning on making a simple map viewer so I can tell how things look in real time.

Right now, I'm playing with making a dumb nbt editor in Qt. See what falls out of my api

[Mr. Analog]

Man this is great, you really get a sense of what's out there, especially how much unused space there is

Also I know where that one block of diamond is haha

[Thorin]

Thanks for continuing to work on this.  Someday we'll be able to clean our map up

One block of diamond, eh?  I think I know where the blocks of lapis lazuli are, as well.

And over 57% of the map is stone, even including air and water blocks!  So we'll be building stone everything for quite a while yet.

And there's still nearly 1.3 million diamond ores to find, with Fortune III on your pick that's about 5 million diamonds!

[Mr. Analog]

Unfortunately most of them are under the ocean haha

[Tom]

Here's a little something I whipped up as a test, and just so I have a decent cross platform nbt editor...



Took a few hours, but I've gotten rusty with Qt especially the model api it has.

oh, I should probably tell you exactly what that is... As I said, its an nbt editor, or more specifically a minecraft .dat editor. The file loaded in that picture is sevalecan's player.dat file