Hey, I just wanted to ask how you acquired all of the logs from logs.tf. Did you use a dump? If so, which one did you use? I found this page with some links to data dumps: https://github.com/tf2-software-enthusiasts/awesome-tf2-devs?tab=readme-ov-file, but I don’t know how up to date that information is since the last commit was two years ago. I also don’t have the connection speed to casually download hundreds of gigabytes of data.

For my project, I would ideally love to have a large collection of the newest logs the bigger, the better and fetch older logs on demand.

Hello, I wanted to ask how much data is currently stored on logs.tf. I'm working on a software project that utilizes logs.tf, and it would be very helpful to know whether it is feasible to essentially store all of logs.tf in my own database in order to get around the rate limiting, or if I should just stick to storing only crucial data for each log in my database on demand.

I keep geeting HTTP Server response 429 when trying to get the json from the logs

This is my method that makes a request to the Logs.tf api, its written in java. Currently i basically get a List of Logs from a player(with having the limit set to 100, it works with having the limit set to 50 though) using the http://logs.tf/api/v1/log?player=Z&limit=X link and then parsing them one by one. First I Call this method to get the JSON and then a diffrent one to parse it, but around halfway through I get a 429 with the lmit set to 100.
My question is: is the Requestlimit the problem here or is my method somehow wrong?

    //Gets the JSON String from a logURL
    String getLogJson(URL logURL){
        StringBuilder content = new StringBuilder();
        try {
            //Connect to Logs.tf and request a Log
            HttpURLConnection con = (HttpURLConnection) logURL.openConnection();
            con.setRequestMethod("GET");

            //Reads the log JSON
            BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
            String inputLine;
            while ((inputLine = in.readLine()) != null) {
                content.append(inputLine);
            }

            //closes the connection
            in.close();
            con.disconnect();

        } catch (IOException e) {
            System.err.println("Connection Error");
            e.printStackTrace();
        }

        //return the full json String
        return content.toString();
    }