BrickSync now in error after last BrickOwl site update

BigBBricks

Anyone else seeing an error occur in BSync after the 600 calls/hour API limit change? I cannot even get BO and BL to sync anymore due to exceeding the limit from normal day-to-day activity. I've alerted Lawrence to the issue, just wondering if any others are experiencing issues.

Big B

Lawrence

As the limit seems to be getting hit by normal Brick Sync users, I've doubled it for now to see how things go. I apologise for the inconvenience.

BigBBricks

Thanks for the quick assist Lawrence, I am able to sync now.

Stragus

I haven't heard of a problem from other users. Lawrence told me about the new 600/minute limit, but I didn't expect any problems.

BrickSync uses a single socket with HTTP pipelining (up to 8 requests in queue). The requests appear to be processed serially by BrickOwl, each taking 200-250ms. BrickSync then pipelines another request as soon as a reply is received, keeping the "in-flight" queue full. So, I had expected at most 300 calls per minute.

I can look into adding a rate limit though.

BigBBricks

I'm guessing I may be the exception due to high lot count in the store

Stragus

Ah yes, I see... Each ID lookup takes about 120ms, and BrickSync will issue a lookup for all unknown BOIDs every time you blaster on/off.

I think I had originally assumed people would have a dozen (or so) unknown BOIDs, at least for a couple days, until the information is submitted and added to the catalog.

I now know the reality is different :), hundreds of IDs can remain unmatched for years, stuff like minifig parts (torso, etc.) or stickered parts. These failing lookups should probably have an incrementing timer so it's not attempted again for a while.

BigBBricks

560 still unresolved lots here though I only run 'owlresolve' every couple of weeks

bricksinbins

Seems I'm running into this also. Trying to do a sync for the very first time so no doubt lots of IDs to resolve. Nothing is working... Not good.

bricksinbins

This is what I get...

INFO: Resolving BOIDs for all unknown BLIDs.
WARNING: BrickOwl HTTP Error - Saving server reply (288+0 bytes) at path "data/errors-2017-06-05/00000.txt".
WARNING: Bad reply from server, trying again shortly...
WARNING: BrickOwl HTTP Error - Saving server reply (288+0 bytes) at path "data/errors-2017-06-05/00001.txt".
WARNING: Bad reply from server, trying again shortly...
ERROR: Too many connection errors, giving up.
ERROR: Fatal error encountered.
ERROR: Exiting now.

Stragus

Check the error reports, such as "data/errors-2017-06-05/00000.txt". See what BrickOwl says.

It's probably related to the API key or its permissions.

bricksinbins

I'm not able to test it right now, but all permissions are enabled except the last two, Access- and Edit BrickOwl Catalog. Access BrickOwl Catalog I can't even enable, it's grayed out altogether.
Thinking about it, isn't that one needed to resolve ID's for example?

Stragus

You shouldn't need catalog access just to resolve IDs. Feel free to look at the actual error, let's see what it says instead of guessing...

Stragus

If anyone is getting error reports where BrickOwl says this:
HTTP/1.0 429 Too Many Requests

Edit your BrickSync configuration file (bricksync/conf/bricksync.conf.txt), change this:
brickowl.pipelinequeue = 8;
to that:
brickowl.pipelinequeue = 1;

We'll need a better solution though...

bricksinbins

Things worked the next day so I'm not sure what the problem was.
However, I'm running some owlsubmitblid and on some BOID/BLID pairs I keep getting this each time:

14:50:57 WARNING: BrickOwl HTTP Error - Saving server reply (766+51 bytes) at path "data/errors-2017-06-07/00004.txt".
14:50:57 WARNING: Bad reply from server, trying again shortly...
14:50:57 WARNING: BrickOwl HTTP Error - Saving server reply (766+51 bytes) at path "data/errors-2017-06-07/00005.txt".
14:50:57 WARNING: Bad reply from server, trying again shortly...
14:50:57 ERROR: Too many connection errors, giving up.

whereas on others it works fine.

Stragus

I think the command "owlsubmitblid" requires the API key permission "Edit Brick Owl Catalog", just check it in your BrickOwl profile.

And check the error report to make sure the problem isn't something else.

bricksinbins

That was copied from the log file. Some BLID submits work fine so permissions should be ok but for some I get that error every time.

bricksinbins

In order to not leave any loose ends I'll admit that the problem with trying to submit BLIDs was my error. It turned out I had mistakenly copied and used the Design ID instead of the BOID. Problem solved.