+- Cuberite Forum (https://forum.cuberite.org)
+-- Forum: Cuberite (https://forum.cuberite.org/forum-4.html)
+--- Forum: Development (https://forum.cuberite.org/forum-13.html)
+--- Thread: Web pages in separate repo (/thread-1767.html)
RE: Web pages in separate repo - xoft - 02-15-2015
What is that robots.txt for, exactly?
RE: Web pages in separate repo - Creaprog - 02-15-2015
The robots.txt ban the robots sniffing the websites.
RE: Web pages in separate repo - xoft - 02-15-2015
And why would we want to ban robots?
RE: Web pages in separate repo - Creaprog - 02-15-2015
Because the robots bad use hugely bandwidth.
RE: Web pages in separate repo - xoft - 02-15-2015
They use bandwidth in order to bring more people to us by showing us in their search results. I don't see anything bad about it, the bandwidth usage is minimal, especially with our single-page site that has almost no links. Additionally, the bandwidth is provided by GitHub and they should be the ones to worry, not us.
RE: Web pages in separate repo - sphinxc0re - 02-15-2015
He's not talking about bots like the Google bot which doesn't generate as much traffic. He's talking about bad bots which scan the webpage in order to make it slow.
RE: Web pages in separate repo - xoft - 02-15-2015
Oh yeah, and a text file is gonna stop those bots, yeah, right
RE: Web pages in separate repo - sphinxc0re - 02-16-2015
Yes... it's like the 5-second-rule where all the bacteria is held up for 5 seconds by the major bacteria saying: "Burrgrgr.. we have to ... burggrg ... wait 5 ....burggrg ... seconds .. bugrgr..." before they start trying to infect what you just dropped on the floor
I just researched a bit. In fact, Github has it's own Robots.txt for the whole webpage and other mechanisms for rejecting bots.
RE: Web pages in separate repo - xoft - 02-22-2015
I have moved the pages to a separate repo: https://github.com/mc-server/mc-server.github.io
The change didn't even need to involve FakeTruth setting anything up for the domain, it just worked out of the box
RE: Web pages in separate repo - NiLSPACE - 03-04-2015
I'm wondering if the mc-server.org supports PHP. If so we might be able to make it show the actual MC version of the build instead of manualy changing it. It's not big, but could fix confusion if we forgot to update the site upon a new Minecraft release.