Indexed JSON documents (file system)
Folks, back to work. I'm trying to convince some colleagues that they should persist "reports" from a product as JSON instead of a fiddly custom text format. Each "report" looks a lot like a self-contained document to me, with a few sections containing the parameters, layout and data of the report. I'm pitching that JSON is standard, readable by everything, change tolerant, and ... most importantly ... databases like Mongo, Cosmos, Dynamo, etc easily store and index JSON documents. They could stuff their reports into a document db and get the indexing, speed and robustness for nothing. However, some customers will want to store all the JSON reports in the file system, so I'm wondering if there is some library that indexes and manages local JSON documents. If I was younger I'd write one myself, but I'll bet someone's already done something similar. Any ideas anyone? I'm searching. I think the latest SQL Server versions do this, but it might be a bit of a heavyweight option. *Greg*
I found this: https://github.com/ketanip/dbjson hope it helps. regards, Preet, in Auckland NZ On Fri, 2 Sept 2022 at 11:20, Greg Keogh via ozdotnet <ozdotnet@ozdotnet.com> wrote:
Folks, back to work.
I'm trying to convince some colleagues that they should persist "reports" from a product as JSON instead of a fiddly custom text format. Each "report" looks a lot like a self-contained document to me, with a few sections containing the parameters, layout and data of the report. I'm pitching that JSON is standard, readable by everything, change tolerant, and ... most importantly ... databases like Mongo, Cosmos, Dynamo, etc easily store and index JSON documents. They could stuff their reports into a document db and get the indexing, speed and robustness for nothing.
However, some customers will want to store all the JSON reports in the file system, so I'm wondering if there is some library that indexes and manages local JSON documents. If I was younger I'd write one myself, but I'll bet someone's already done something similar. Any ideas anyone? I'm searching. I think the latest SQL Server versions do this, but it might be a bit of a heavyweight option.
*Greg* -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
And also this: https://github.com/SleekwareDB/sleekwaredb regards, Preet, in Auckland NZ On Fri, 2 Sept 2022 at 12:28, Preet Sangha <preetsangha@gmail.com> wrote:
I found this: https://github.com/ketanip/dbjson hope it helps.
regards, Preet, in Auckland NZ
On Fri, 2 Sept 2022 at 11:20, Greg Keogh via ozdotnet < ozdotnet@ozdotnet.com> wrote:
Folks, back to work.
I'm trying to convince some colleagues that they should persist "reports" from a product as JSON instead of a fiddly custom text format. Each "report" looks a lot like a self-contained document to me, with a few sections containing the parameters, layout and data of the report. I'm pitching that JSON is standard, readable by everything, change tolerant, and ... most importantly ... databases like Mongo, Cosmos, Dynamo, etc easily store and index JSON documents. They could stuff their reports into a document db and get the indexing, speed and robustness for nothing.
However, some customers will want to store all the JSON reports in the file system, so I'm wondering if there is some library that indexes and manages local JSON documents. If I was younger I'd write one myself, but I'll bet someone's already done something similar. Any ideas anyone? I'm searching. I think the latest SQL Server versions do this, but it might be a bit of a heavyweight option.
*Greg* -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
I found this: https://github.com/ketanip/dbjson hope it helps.
If I had to do it myself I'd probably do something similar, maybe use SQLite to hold an index of the properties of JSON documents. I would have to walk every document's tree (fragile) and the hierarchy of properties would have to be represented as relational tables (hacky, not meant for that). Still thinking -- *Greg*
At least in Cosmos, there’s a good story with the indexing policy that’s very configurable. (Including fully automated if that’s what you need) Regards, Greg Dr Greg Low 1300SQLSQL (1300 775 775) office | +61 419201410 mobile SQL Down Under | Web: https://sqldownunder.com<https://sqldownunder.com/> | About Greg: https://about.me/greg.low From: Greg Keogh via ozdotnet <ozdotnet@ozdotnet.com> Sent: Friday, 2 September 2022 10:52 AM To: ozDotNet <ozdotnet@ozdotnet.com> Cc: Greg Keogh <gfkeogh@gmail.com> Subject: Re: Indexed JSON documents (file system) I found this: https://github.com/ketanip/dbjson hope it helps. If I had to do it myself I'd probably do something similar, maybe use SQLite to hold an index of the properties of JSON documents. I would have to walk every document's tree (fragile) and the hierarchy of properties would have to be represented as relational tables (hacky, not meant for that). Still thinking -- Greg
At least in Cosmos, there’s a good story with the indexing policy that’s very configurable. (Including fully automated if that’s what you need)
Indeed! We're using CosmosDB elsewhere in the suite, so I'm going to make a demo on the weekend where the "reports" are saved in Cosmos. The total coding required should be trivial, but the advantages are enormous. I have to find out just how many customers may want to store their reports locally, maybe it's not an issue -- *GK*
participants (3)
-
Dr Greg Low -
Greg Keogh -
Preet Sangha