Hello!
To use the forum, login or register above.
(If you are already logged in above, please click here)
What are the legal/technical issues preventing BO/BL synchronization?
I have read the posts related to the tragic passing of Richard Morton, author of BrickTrak, while he was working on a new BrickOwl-focused version. Some posts in related threads mention the severe legal barriers put by BrickLink to prevent usage of their data. After actually reading the Terms of Service, it's even worse than I had thought.
Since I plan to start selling extra parts later, I have written my own little inventory management software, working with BrickStore's .bsx files, using (and contributing to) BrickStore's cache, fetching data from BrickLink (BrickStore's source code was such a mess that it seemed easier to write something working alongside it).
I would like to consider writing a simple command-line, open-source tool that would keep an inventory in sync with both BO and BL. A seller would run the software on their own computer, connections would be kept open to both APIs 24 hours a day, to receive orders and subtract from inventory. The command-line software would use commands such as "add newstuff.bsx" (.bsx=BrickStore file) to modify its internal inventory, propagating the changes to both BO and BL.
I believe this would be compatible with BL's Terms of Service, especially point 8 : "You acknowledge and agree not to distribute, disclose, upload, or transfer to any third party any content or data you receive from or which is displayed on the Site, which includes but is not limited to the inventory data files in the form of XML, CVS, or Tab-Delimited format, except when this is required for the operation of your BrickLink store." Did I miss any other legal issue?
Another issue is the differences between the BO and BL catalogs. Searches for some of BrickLink's item IDs on BrickOwl fail with no results, but would the API be able to understand BL's item IDs? Could the data from BrickOwl orders include BrickLink's item IDs somewhere? Otherwise, we'll need some translation layer sitting in the middle... Urggh, does the data for translation of IDs Bo<->BL exist somewhere?
I don't really do GUI so this would be command-line, neither pretty nor user-friendly software (in its first versions anyway). Do I miss any legal/technical issues that would prevent sellers from running such synchronization software on their own computer? (no third party involved)
I'm sure many BO users don't care about the other site, but such a tool would help big sellers (and their buyers!) to migrate. I can't be an advocate of open-source software, perhaps almost a Linux zealot, while supporting their business model. I would feel better never placing an order on BrickLink again.
Since I plan to start selling extra parts later, I have written my own little inventory management software, working with BrickStore's .bsx files, using (and contributing to) BrickStore's cache, fetching data from BrickLink (BrickStore's source code was such a mess that it seemed easier to write something working alongside it).
I would like to consider writing a simple command-line, open-source tool that would keep an inventory in sync with both BO and BL. A seller would run the software on their own computer, connections would be kept open to both APIs 24 hours a day, to receive orders and subtract from inventory. The command-line software would use commands such as "add newstuff.bsx" (.bsx=BrickStore file) to modify its internal inventory, propagating the changes to both BO and BL.
I believe this would be compatible with BL's Terms of Service, especially point 8 : "You acknowledge and agree not to distribute, disclose, upload, or transfer to any third party any content or data you receive from or which is displayed on the Site, which includes but is not limited to the inventory data files in the form of XML, CVS, or Tab-Delimited format, except when this is required for the operation of your BrickLink store." Did I miss any other legal issue?
Another issue is the differences between the BO and BL catalogs. Searches for some of BrickLink's item IDs on BrickOwl fail with no results, but would the API be able to understand BL's item IDs? Could the data from BrickOwl orders include BrickLink's item IDs somewhere? Otherwise, we'll need some translation layer sitting in the middle... Urggh, does the data for translation of IDs Bo<->BL exist somewhere?
I don't really do GUI so this would be command-line, neither pretty nor user-friendly software (in its first versions anyway). Do I miss any legal/technical issues that would prevent sellers from running such synchronization software on their own computer? (no third party involved)
I'm sure many BO users don't care about the other site, but such a tool would help big sellers (and their buyers!) to migrate. I can't be an advocate of open-source software, perhaps almost a Linux zealot, while supporting their business model. I would feel better never placing an order on BrickLink again.
Powered by Vanilla
Comments
To answer your detail question about mapping, BO maintains mapping as part of its open catalog license. However, you are dependent on the maintenance/timing of that mapping and therefore do have to deal with items in limbo (listed in one store and not the other). I have about 15 inventory items waiting for updates on BO to allow them to appear.
The BO API does not yet have a push feature, so you need to listen for order emails. The BL API did add a push feature, but I'm not sure how reliable it is. Syncing order events to decrease inventory in the opposite store is the easy part.
The deeper challenge is dealing with non-order inventory changes, such as parting out a set, manually updating an inventory count or adding a new lot, or refund/return/cancel automation that restores order items to inventory. Since there is no way to prevent these actions in either BO or BL, you have to have some way to reconcile.
For our stores, I have the email listeners automated, but I still have to do a manual full BSX import every day or two. Theoretically that could be automated, but doing it manually allows me to see the list of BL parts that aren't matching BO catalog, and to submit catalog updates for them.
Good luck!
That BO maintains a mapping of the BL catalog is great news. Is there a reason why that is not exposed while searching? Let's say, gs005, some minifig found under 688255 on BrickOwl.
I agree with non-order inventory changes being more tricky, hence why I suggested that any such change would involve a command "add newstuff.bsx", manually importing a custom list of parts defined in BrickStore (including negative amounts).
The lack of Push feature on BrickOwl is an issue Admin could fix (unless he doesn't mind if we poll for new orders every minute). I would rather avoid the email route entirely.
I'll work on this once I start selling, perhaps in a month or so. I have a bunch of BrickStore/BrickLink code already written, what's missing should be pretty straightforward.
Thanks!
/v1/order/items list various IDs, such as peeron_id, but nothing apparently related to BrickLink. Is it just because someone yet has to do manual item edits to add the BL_IDs? That's a lot of items missing BrickLink IDs...
We could automatize the process by finding, for each assembly on BrickOwl (let's say, a minifig), the assembly on BrickLink that has exactly the same inventory. I could write that code, it's a good thing the API lets us send catalog edits!
Is that the way to proceed, or I'm missing something related to BO<->BL translation?
A table to translate color IDs would also be nice, but that's simple enough to build manually.
http://www.peeron.com/inv/colors
Ok, I see the link now down at the bottom. Maybe move it higher or make it more prominent? That would have really helpful to find sooner (spent a lot of time trying to verify colors of parts I was buying, sorry, newbie here). Is there any kind of "new buyers guide"?
BrickOwl doesn't output BrickLink IDs anywhere under any shape or form. If we want to do a reverse translation BOID->BL_ID, we can store custom data in the "external ID" field of inventory lots through API commands, and we can get that back when listing orders as the "external_lot_ids" field. That works for me.
Regarding submitting missing BL_IDs to BrickOwl... Finding an assembly (minifigure/other) on BrickLink that matches a given list of parts wouldn't be too hard, but I'm finding many minifigures aren't yet listed as separate assemblies on BrickOwl (rather, the parts are all mixed in the set's inventory). That's more tricky, I don't think that can be reliably automated...