iDB
(short for item database)
Description
- Windows app + companion web interface
- Drop, transfer, or pickup items
- Virtual databases to separate out items
- Item images (as of September 2019!)
- Reseller support (as of October 2019!)
- Auto‐refresh characters before they expire
- Automated socketing (not Larzuk, but inserting gem/rune/jewel into bases) / cubing feature
- Integration with WorldIsMine's AutoShop, Laz's ShopDrop and my SimpleShop
- Integration with WooCommerce for order delivery
- Integration with NeutronPlus & PermIt
Architecture
Hardware Requirements
- Windows PC with Windows 10 or Windows Server 2016+, Windows Server is highly recommended due to connection limitations for IIS
- 4 core processor (preferably 8+ core AMD), more cores is best way to improve performance for running many clientless bots
- Good network card
- 16 GB or more of memory
Kolbot Integration
LinkShopDrop Integration
LinkDelivery Integration
LinkCodes
Codes are like products in a web shop (eg. Fortitude - Archon Plate - Ethereal - Bugged - 30% All Res, 1.5 HP/Lvl, 3552 Defense). They allow you to categorize your items and search for them easily. A Code consists of an ID (0 to 999,999), a description, a type (eg. Archon Plate) and a match/pickit (eg. Quality = Runeword & Name = Fortitude & Eth & Ebugged & AllResistances = 30 & LifePerLevel = 1.5 & Defense = 3552).How to Write a Match/Pickit
LinkSpecial
There are 4 types of special codes. These allow you to create packs containing other codes! They can be multiple levels deep as well (A contains 1 B and 2 C's, and C contains 3 D's and 1 E). Special codes have a match format of [id]:[count],[id]:[count],.. (eg. 1:2,5:4,8:1).Type | Description |
---|---|
Special | A plain pack |
SpecialStrict | A pack that fails immediately if only partially available, the app will do its best to make sure this pack is delivered in its entirety |
SpecialAny | Quantity on each ingredient is ignored, this pack resolves to the first code that is in stock (on the given realm/database) |
SpecialAnyEven | Quantity on each ingredient is ignored, this pack resolves to the code that has the highest stock (on the given realm/database) |
Exporting
Drag your db.sqlite file onto dump-Codes.bat and it will export to a Codes-*.sql file.- This can be done even while iDB is running.
- If you're about to do an import, it is always a good idea to back up your current set of codes.
Importing
Advanced users only!Drag your *.sql file onto import-Codes.bat and it will import to db.sqlite.
After running iDB you will see a screen like this for a few minutes. If you have millions of items it can take up to an hour to reprocess all.
- You should only do this if iDB is closed.
- Since this operation does a full replacement, it must be full set of Codes, not just a partial set.
- Be very careful not to change IDs of Codes that you are using with ShopDrop/AutoShop.
- Be very careful not to mess up the formatting of your .sql file.
Images
Features
- Item level is appended to the title
-
Unidentified uniques and sets will show the correct identified name (if possible):
-
Replenish over time stats (replenish quantity, etc) have the exact details:
- Per level stats (life per level, etc) have the exact details:
-
Ethereal bugged items will have (Ebugged) appended to their title:
-
Upped items will have (Upped) appended to their title:
Hotkeys
While the image is showing, you can do CTRL+S to save the image to hard-drive.While the image is showing, you can do CTRL+C to copy the image to clipboard.
While the image is showing, you can do CTRL+U to upload image to Imgur and copy link to clipboard.
Player Considerations
Be wary of items that potentially give +4 or more skills to non-owner classes, like Widowmaker which can spawn with up to +5 to Guided Arrow. In-game and in other kolbot-based droppers you might see that this Widowmaker has only +3 to Guided Arrow:Databases
Databases are a virtual separation of items that allow the following configurations:- Priority for operations that span multiple databases (order preference, etc)
- Whether or not web shop order delivery should be able to access
- Whether or not ShopDrop order delivery should be able to access
- Whether or not the characters should be auto-refreshed
- Different limit IDs for using different keys & proxies via rDB
- Special reseller access via iDB-Web
Processors
Databases are a virtual separation of items that allow the following configurations:Name | Description |
---|---|
Jobs | Changes jobs from inactive to active (while obeying queue limits such as LimitPerRealm and also not activating a job for an account that is already active) |
API: TCP | Turns on a listener for PlainSocket network connections such as iDB-Web, PermIt, NeutronPlus, Kolbot, etc |
API: HTTP | Turns on a listener for WebSocket network connections such as iDB-Web, PermIt, NeutronPlus, Kolbot, etc |
API: Allow New Drops | If unchecked, drop requests will be responded with "Drops are temporarily disabled (service will restart soon)." (does not affect ShopDrop requests) |
AutoRefresh | Toggles the processor that checks for expiring characters and queues jobs to refresh them |
Delivery | Toggles the processor that communicates with the web shop for order delivery |
ShopDrop | Toggles the processor that communicates to the ShopDrop app (only controls iDB -> ShopDrop, to control ShopDrop -> iDB communication use the API: TCP processor) |
Converting to non-ladder
When a ladder reset is coming close you should follow these steps.-
Uncheck all processors.
-
Abort all jobs.
-
Start a manual backup.
-
A new set of files timestamped with the current date/time will be created. Wait a few minutes for the *-shm and *-wal files to disappear.
-
In iDB's Items window, click on Convert to NL.
Recovery
Let's say you accidentally delete all your items and you're wondering how to restore them.-
Uncheck all processors.
-
Abort all jobs.
- Close iDB.
-
Open IIS and stop the website pool.
- Rename C:\Apps\iDB\db.sqlite to C:\Apps\iDB\db.sqlite.bak because it is a bad database now (or delete it).
- Rename latest backup file (like C:\Apps\iDB\db-2020-03-19-14.40.sqlite) to C:\Apps\iDB\db.sqlite.
- Open IIS and start website pool.
- Start iDB.
Important Notes
- Settings are stored in C:\Apps\iDB\iDB.xml.
- Databases, codes, items, etc are stored in C:\Apps\iDB\db.sqlite.
- Every 2 days your db.sqlite and your web logins are backed up to my server.
Changelog
2023-11-12: v1.0.0.735
-
ADD
Added
AllowJoiningPublicGames for trashers (default is true)
2023-03-26: v1.0.0.733
-
ADD
Added
AllowJoiningPublicGames for loggers (default is true)
2021-08-19: v1.0.0.731
-
ADD
Added
ShouldEnterGame for refreshers (default is false)
2021-07-23: v1.0.0.730
- UPD Improved updating of character expirations
2021-07-20: v1.0.0.729
- ADD Added button on Items window to delete expired characters (also respects realm/database filters)
2021-06-17: v1.0.0.726
- ADD Added automatic job prioritization for Ladder
2021-06-12: v1.0.0.723
- UPD Changed behavior to warn instead of deleting when encountering mismatched character core/ladder/expansion
2021-06-11: v1.0.0.722
- ADD Added Move to Different Database button for currently filtered items
2021-06-08: v1.0.0.720
- FIX Fixed a bug regarding reactivation of ShopDrop or AutoShop after a period of inactivity
2021-06-03: v1.0.0.719
- UPD If wait-for-pickup is on, droppers will wait before dropping carry-only-1 items (annis, torches, gheeds, cube, etc) if there is already one on the ground
2021-05-29: v1.0.0.717
- FIX Fixed bug with JSP shop processor not reporting failures for out of stock products
- UPD Improved logging for JSP shop
2021-05-27: v1.0.0.715
- FIX Minor fixes for builder
2021-05-26: v1.0.0.714
- UPD Updater improvements
- FIX Minor fixes
2021-05-18: v1.0.0.699
- ADD Added support for mappings in SimpleShop
2021-05-17: v1.0.0.696
- ADD Added new Deletion.log for account and character deletions
- FIX Fixed expiration update bug (when sent from PermIt, etc)
- UPD Updated expiration updates to delete missing characters (when sent from PermIt, etc)
2021-04-29: v1.0.0.689
-
ADD
Added
AutoRetryOrderOnGameFullBugged (default is true)
2021-04-29: v1.0.0.688
- UPD Updated web drop messages
2021-04-22: v1.0.0.684
-
UPD
Set
DisableWest to default to false
2021-04-17: v1.0.0.681
-
ADD
Added new option
DisableWest (default is true) which disables drops from iDB-Web, Refreshers and Loggers for the West realm
2021-04-13: v1.0.0.680
- UPD Adjusted abort job options to focus on refresher jobs
2021-04-12: v1.0.0.679
- FIX Potential bug fix for errors on job ending
2021-04-10: v1.0.0.678
- ADD Added builder request # to title
2021-04-09: v1.0.0.677
- ADD Added ability to trash items (drop in public game)
2021-04-05: v1.0.0.674
- ADD Added character details to summary for pickup jobs
2021-04-05: v1.0.0.673
- ADD Added ability for loggers to attempt to join public games
2021-04-05: v1.0.0.672
- ADD Added manual pickup game/password to module title
2021-03-25: v1.0.0.666
- ADD Added WaitForPickupIfUnitPriceGreaterThan for SimpleShop
2021-03-23: v1.0.0.665
- ADD Added Delete Items Matching Current Filters button on items tab
2021-03-18: v1.0.0.664
- FIX Fixed decimal support for SimpleShop
2021-03-18: v1.0.0.663
- ADD Added support for multiple SimpleShops
2021-03-07: v1.0.0.659
- FIX Fixed decimal support for JSP Shop
2021-03-02: v1.0.0.658
- ADD Added item image borders
2021-03-02: v1.0.0.657
- UPD Performance improvement for searching items
2021-02-28: v1.0.0.656
- ADD Added IP address to API logs
2021-02-28: v1.0.0.655
- UPD Improved responsiveness of Items and Codes windows
- UPD Builder now reports what ingredients were missing if it fails
- UPD Pack editor sorts by ID
- UPD Double-clicking in pack editor will add the code now
2021-02-23: v1.0.0.652
- FIX Fixed using presets in Transfer mode
2021-02-23: v1.0.0.650
- UPD Allow comments in SimpleShop definitions
2021-02-21: v1.0.0.649
- FIX Fixed sorting by Name in Items window
2021-01-28: v1.0.0.642
- ADD Added pickup/transfer presets
2021-01-28: v1.0.0.641
-
ADD
Added
LimitGlobal for modules -
UPD
LimitPerOrder andrDBIDDeliveryOverride are removed for certain modules
2021-01-25: v1.0.0.638
- ADD Added SimpleShop
2021-01-24: v1.0.0.632
- UPD Updated to infinite retries for sending order results
2020-12-29: v1.0.0.629
- ADD Added handling for multiple server disconnects from lobby
2020-12-20: v1.0.0.628
- FIX Quick fix for SpecialAny missing from db
2020-12-15: v1.0.0.626
- UPD Massive log performance improvement
2020-12-15: v1.0.0.624
-
ADD
Added
JSPShop*-ConnectionTimeout for auto-disabling JSP shop updates if the app is having trouble connecting (for example, ShopDrop/AutoShop is not running but the iDB processor is left on) - UPD Stock updates are delayed and batched together to help with Log performance
2020-12-09: v1.0.0.619
- UPD Re-indexed accounts & characters case-insensitively
- UPD Upserting accounts & characters uses SQLite's ON CONFLICT feature for performance
2020-12-09: v1.0.0.618
- ADD Added Get # of Items Matching Current Filters button on Items window
- UPD Performance improvements during Log operations and more
- UPD Removed prioritized db queue
- ADD Added logging for Log operations to help track any slowness
2020-12-08: v1.0.0.615
- ADD Added visualization of db vacuuming
- UPD Prevented stacking of clean-and-reset operations
2020-12-07: v1.0.0.614
- UPD Deleting a database works even when there are items attached now
2020-12-06: v1.0.0.613
- UPD Improved jsp file import
2020-12-04: v1.0.0.611
- FIX Fixed bug where backup got stuck
2020-12-04: v1.0.0.610
- UPD Improved tracked item hash conversion to NL
2020-12-03: v1.0.0.608
- ADD Added message that manual backup completed
2020-11-28: v1.0.0.607
- FIX Fixed to limit Log operations
2020-11-28: v1.0.0.606
- ADD Added limit to number of backgrounded Log db operations
2020-11-24: v1.0.0.605
- ADD Added special code recursion detection
2020-11-11: v1.0.0.603
- FIX Fixed counts for SpecialAny and SpecialAnyEven codes
2020-11-11: v1.0.0.602
- UPD Improved game re-use by loggers
2020-11-09: v1.0.0.601
- FIX Fixed bug with SpecialAnyEven codes
2020-11-06: v1.0.0.593
- ADD Added date to tracked item hashes
- ADD Added tracked price and date to Items window
- UPD Items column state is persisted now
2020-11-04: v1.0.0.592
- ADD Added support for Hash in pickits
2020-10-28: v1.0.0.590
- ADD Added support for SpecialAnyEven codes
2020-10-13: v1.0.0.581
- ADD Added prioritization system for db queue (Log operations are low priority)
2020-10-04: v1.0.0.575
- ADD Added SystemOffline and DropFinished packets for ShopDrop
2020-10-04: v1.0.0.574
- UPD Improved performance when jobs are ending
2020-09-30: v1.0.0.572
- UPD Parallelized incoming API requests to improve responsiveness
- FIX Fixed display order of jobs on main window
2020-08-12: v1.0.0.568
- UPD Updated overhead messages to every 3 seconds
2020-07-28: v1.0.0.567
- FIX Fixed picker switching to create after joining fails
2020-07-28: v1.0.0.566
- ADD Added rDB ID override for manual pickup or transfer jobs
2020-07-12: v1.0.0.563
-
ADD
Added
PauseAndAbortAllOnRealmGoingDown
2020-06-28: v1.0.0.559
- FIX Automatic detection and fixing of orders stuck in processing state
2020-06-27: v1.0.0.558
-
ADD
Added
LimitPerGameJspShop for JSP shop droppers
2020-06-15: v1.0.0.557
-
ADD
Added
rDBIDJspShopOverride for droppers
2020-06-15: v1.0.0.556
- FIX Fixed broken job queue behavior
2020-06-11: v1.0.0.555
- ADD Added database to the confirmed drops in JSP shop logs
- ADD Added failed drops to JSP shop logs
2020-06-11: v1.0.0.552
- UPD Made overhead pause message persistent
2020-06-09: v1.0.0.552
- ADD Added overhead message for pause / resume
2020-06-08: v1.0.0.550
-
ADD
Added
ShopDrop-WaitForPickupMessage andAutoShop-WaitForPickupMessage that is spammed every 5 seconds
2020-06-07: v1.0.0.549
- FIX Fixed queue performance degradation with large number of jobs
2020-06-06: v1.0.0.548
- FIX Fixed bug where multi-tier strict special codes were not processing and aborting correctly
2020-06-06: v1.0.0.547
- ADD Added pause and resume commands for picker
2020-06-05: v1.0.0.545
- UPD Minor logging improvements
2020-06-03: v1.0.0.544
- UPD Module terminates if lobby thread crashes
2020-06-03: v1.0.0.543
- FIX Fixed bug where droppers didn't abort if builder never made it into a game
2020-06-02: v1.0.0.542
- ADD Added account and character creation to builder
- UPD Allow replacement builders to bypass queue restrictions
- FIX Fixed droppers getting stuck waiting for a builder that aborted
- FIX Added retries to cube transmuting
- FIX Fixed a scenario where items were staying locked after an order
2020-05-25: v1.0.0.541
- UPD Added checks to prevent recursive packs from being created
2020-05-25: v1.0.0.540
- FIX Improved error handling if item fails to parse
2020-05-03: v1.0.0.539
- FIX Fixed item graphics with spaces
2020-04-10: v1.0.0.537
- FIX Fixed SystemOffline message for AutoShop
2020-04-10: v1.0.0.536
- UPD Deleting items is more careful with cache clearing
2020-04-08: v1.0.0.535
- ADD Added AutoShop support
2020-04-08: v1.0.0.533
- FIX Fixed db import
2020-04-06: v1.0.0.529
- FIX Fixed pickits with CrescentMoon name filter
2020-04-02: v1.0.0.527
- FIX Fixed event handler memory leak and disposing of more SQLiteCommands
2020-04-02: v1.0.0.526
- FIX Fixed pickits with *MaxDamage stat
2020-03-28: v1.0.0.523
- FIX Fixed hotkey interference on some windows
2020-03-28: v1.0.0.522
- FIX Fixed issue when importing ShopDrop data with different name or thumb
2020-03-24: v1.0.0.521
- ADD Added pack editor for Special codes
2020-03-16: v1.0.0.519
- ADD Added support for SpecialAny codes
2020-03-12: v1.0.0.518
- ADD Added builder support for Special type
2020-02-28: v1.0.0.516
- FIX Orders for hashes obey order-delivery-access flag
- FIX Fixed db persistence of failed items on orders
- UPD Synchronized job termination
2020-02-25: v1.0.0.514
- FIX Logger now only tries to re-use game if game creation was successful
2020-02-25: v1.0.0.513
- FIX Fixed caching of special codes even after deletion
2020-02-23: v1.0.0.511
- FIX Fixed another graphic bug
2020-02-21: v1.0.0.510
- FIX Fixed background backup/vacuum
2020-02-21: v1.0.0.508
- UPD Backup/vacuuming happens in background now
- UPD Updated SQLite to 1.0.112.0
2020-02-19: v1.0.0.507
- FIX Added delay after trading to allow for replacement items to be received
2020-02-12: v1.0.0.506
- UPD Updated ShopDrop window title
2020-02-09: v1.0.0.505
- FIX Fixed droppers get stuck in infinite loop when code not found
2020-02-03: v1.0.0.503
- UPD Builder updates
- ADD Tcp support for gold, imbue, socket
2020-01-21: v1.0.0.502
- FIX Fixed ebugged property
2020-01-19: v1.0.0.498
- FIX Minor builder fixes
2020-01-15: v1.0.0.495
- ADD Started logging gold/imbue/socket quest
2020-01-14: v1.0.0.493
- FIX Fixed thumb for ShopDrops with graphic in pickit
2020-01-13: v1.0.0.492
- ADD Added sync tool for tracked item hashes
2020-01-13: v1.0.0.488
- ADD Added batching for tracked item hash publishing
2020-01-09: v1.0.0.485
- ADD Added batch button for ShopDrop
- FIX Fixed ordering of ShopDrop messages
2020-01-09: v1.0.0.483
- UPD Removed Clear All Codes button
2020-01-08: v1.0.0.481
- FIX Fixed failed codes friendly string for unsupported SKUs
2019-12-28: v1.0.0.476
- UPD Logger should use same game if possible
2019-10-20: v1.0.0.405
- ADD Droppers will terminate other droppers if game does not exist
2019-09-29: v1.0.0.390
- ADD Added item images
2017-02-08: v1.0.0.0
- ADD Initial