Create
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Products
See all
Community resources
Top groups
Explore all groups
Community resources
Learn
Community resources
Events
Community resources

parse all bitbucket repositories from many json files

Sofien Chaabouni
Sofien Chaabouni October 5, 2015

I'm trying to parse all json files of Bitbucket repositories of one use! I took Atlassian's repos as an example because they have more than 300 repos! The problem is that for REST API we can get only 10 repos from one request so I tried to make many requests but I couldn't make it :

<!DOCTYPE html>
<html>
<body>
  <p id="demo"></p>
  <script>
    var arr = [];
    var nextPage = true;
    var num = 1;
    var out = "";

    function httpRequest (theUrl, cb){
      var xmlHttp = new XMLHttpRequest();
      xmlHttp.open("GET", theUrl , true);
      xmlHttp.send( null);
      xmlHttp.onload = function(){
        cb(null, xmlHttp.responseText);
      };
      xmlHttp.onerror = function() {
        cb(new Error("request error"));
      };
    }

    while (nextPage) {
      httpRequest( "https://api.bitbucket.org/2.0/repositories/atlassian/?page=" + num , function (err, data) {
        if (err) {
          arr = [];
        } else {
          arr = JSON.parse(data);
        }
        if (arr.next) {
          num ++;
        } else {
          nextPage = false;
        }
        var i; 
        console.log(arr);
        out += "<h4>"+arr.next+"</h4>"
        document.getElementById("demo").innerHTML = out;
      });
    }
  </script>
</body>
</html>


I always have num = 1 inside httpRequest attributes!

Any help ?

 

thanks

Watch
Like Be the first to like this
649 views

2 answers

1 accepted

Comments for this post are closed

Community moderators have prevented the ability to post new answers.

Post a new question

2 votes
Answer accepted
Vitalii Petrychuk
Vitalii Petrychuk
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 5, 2015

Hi Sofien,
Your example doesn't work because of few things:

  1. XMLHttpRequest is async.

  2. You do use while together with async XHR which freezes browser for a while (until nextPage is set to false).

You can fix it in few ways:

  1. Make your XHR sync - xmlHttp.open("GET", theUrl, false); - which is a bad idea because it blocks the thread until request is competed.
  2. Instead of while use a recursive function that invokes itself when the callback is executed

Please note that you can use pagelen param to load more repos at once e.g. https://api.bitbucket.org/2.0/repositories/atlassian/?page=1&pagelen=100.
Also I'm not sure if your cross-domain request passes.

I hope this will help.

Reply
0 votes
Sofien Chaabouni
Sofien Chaabouni October 6, 2015

thanks Vitalii, still the pagelen attribute have a max of 100 only!

For cross-domain request I had this issue also with django! any suggestion ?

View More Comments
Vitalii Petrychuk
Vitalii Petrychuk
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 6, 2015

Cross-domain request cannot be performed by browser only due to security issues, but on the server side (using python in your case) you should be able to make it without any problems.

Like
Sofien Chaabouni
Sofien Chaabouni October 6, 2015

Yes I used python and django for my addon but I had some problems in deployment and testing it on bitbucket : https://answers.atlassian.com/questions/30942456/answers/30953248?flashId=338941302

Like
Reply
Atlassian logo
Answers Developer Questions
TAGS
AUG Leaders

Atlassian Community Events

    See all events