I'd like to start a discussion on creating an open-source driver database that exposes a public RESTful API. My primary reasons for doing so are:
I have quite a bit of experience with Python(Django) and JavaScript(Angular, Backbone, D3) and need something new to work on. My initial ideas are to have a web-based API, a web client and a CLI client that presents data models including Driver, Manufacturer and Spec. All code would be public available as Github repos.
With that said, I'm all ears for user feedback.
- Platform-specific native apps raise the barrier to entry for users and developers
- Madisound and PE each provide different search interfaces with different functionality, and hopping between them is inefficient and insufficient
- A system that allowed users to compare manufacturer specs to user specs(derived from tests) would be useful
- I would find such a tool useful, and am willing to build it
I have quite a bit of experience with Python(Django) and JavaScript(Angular, Backbone, D3) and need something new to work on. My initial ideas are to have a web-based API, a web client and a CLI client that presents data models including Driver, Manufacturer and Spec. All code would be public available as Github repos.
With that said, I'm all ears for user feedback.
2) After selecting a category, PE includes filters for: nominal diameter, impedance, power, magnet material, cone material, price and brands. While arguably some of the most important filters, they constitute all parameters. After selecting a category on Madisound, a filter is provided for nominal diameter and that's it. I'd like to expose filters for all driver parameters within the API to facilitate more granular queries. Within a web UI, I could default to a limited set of filters, but also provide a mechanism for work with them all.
3) Manufacturer specs provide a good starting point, but they shouldn't necessarily be considered sacrosanct. I think the differences between data from the manufacturers and data collected from other parties is interesting and potentially valuable. I'd like to provide an interface for users to upload their own test results, and then differentiate between manufacturers specs, consumer specs, and perhaps provide averages for users to query against.
Just playing devil's advocate, how would you ensure the user knows how to measure?
There are any number of lists of driver parameters out there.
There used to be a big thiele small database project, but it apparently never went anywhere. After hanging around for a few years, it is gone.
A guy named Bob Stout once made something similar, his LDSG. LDSG Section7 - Woofers
There are any number of lists of driver parameters out there.
There used to be a big thiele small database project, but it apparently never went anywhere. After hanging around for a few years, it is gone.
A guy named Bob Stout once made something similar, his LDSG. LDSG Section7 - Woofers
Firstly, thanks for the feedback. I don't want to work in a vacuum, and am very much interested in the ideas of other DIYers. I built my first speaker just a year ago, and know that I don't have all the ideas for a project like this.
Regarding your questions:
1) I wouldn't. I would probably opt to default to manufacturer's specs for handling a request for a driver, but also provide access to third-party specs. The two categories would be differentiated by something like a boolean field.
2) I poked around a bit on Google and in older threads around here, and that seems to be a recurring theme. In my favor is the open source path, as well as the fact that there are only two major resellers of raw drivers. Writing scrapers to update prices and add new drivers as they appear isn't a terrible amount of work. I'd like to get the code base in place, and then automate away the updating of the data so my involvement after launch is limited to bug fixes and new features.
3) LDSG is similar in its tabular presentation, but I want to provide much more functionality like filtering, search, sorting, and grouping. Grouping would allow a user to persist a list of selected drivers relevant to a task at hand (i.e. Bookshelf Candidates, Boombox Candidates, etc.).
I am certainly not trying to discourage you. If you find it useful, go for it.
Others may be of a similar mind and manufacturers (especially car audio where they sell a lot of raw drivers to end users) might like it as a marketing tool and participate. I don't know what percentage of PE or Madisound or Meniscus's business is consumer vs. commercial.
I know what you mean by sorting and having more fields to sort on. It is surprising though how much of this sort of "properties data" is wrong on many websites for many products. Best buy and microcenter have lots of data on number and type of inputs, outputs, etc... and it is often quite wrong. The longer your table gets, and the more automated the scraping is, the harder it is going to be to maintain integrity.
Others may be of a similar mind and manufacturers (especially car audio where they sell a lot of raw drivers to end users) might like it as a marketing tool and participate. I don't know what percentage of PE or Madisound or Meniscus's business is consumer vs. commercial.
I know what you mean by sorting and having more fields to sort on. It is surprising though how much of this sort of "properties data" is wrong on many websites for many products. Best buy and microcenter have lots of data on number and type of inputs, outputs, etc... and it is often quite wrong. The longer your table gets, and the more automated the scraping is, the harder it is going to be to maintain integrity.
It's been proposed before and done before, and after a couple of years the database starts to deteriorate and the developer pulls the plug.
For example, the database that used to be at thielesmall.com claimed to have over 5000 drivers. And, in fact, it did. But when you eliminated the drivers that had no TS data or that were out of production or where the TS data didn't make any sense, the number dropped to around 2700. That's just how efforts go...drivers become obsolete, people are allowed to enter junk, and nobody wants to clean up the mess. This file has all of the drivers from the thielesmall database, with the "junk" trimmed out on one of the pages.
There is a more recent database from about 3 years ago that included everything from PE, MCM, and the thielesmall.com database. That's in this Excel file.
I had imported all of that data into a Microsoft CE database, which worked well for the .NET-based PSD-Lite design tool. The CE database is a single file that could be "shared" and modified collaboratively using tools that were integrated into PSD-Lite. But very few people took advantage of the integrated TS database, and then Microsoft dropped support for the CE database. So once again...another defunct TS database. I've also got another Excel file that has all of the entries for the *really* old Bullock and White DOS-based design tools, and, of course, there are the embedded databases in the various versions of WinISD.
The thielesmall.com database was actually much more ambitious, in that it attempted to capture additional driver physical parameters such as bolt patterns and depth. But that level of detail is not worthwhile: it is better to provide links to a datasheet repository, which is a whole other challenge. So if you are serious about building a useful TS database, you probably need to work with the vendors to have permanent locations for data sheets so that you can provide the links. Also, the TS database is really only useful for box design, and for a driver database to be more useful there should be associated measurement data, or at least links to the measurement data. Here is the link to the measurement database for PSD-Lite: Directory Listing of /Reference/FRD_data/
I don't mean to sound discouraging, but I think another TS database is a waste of time. For the next iteration of PSD-Lite I'm just going to keep it simple and provide a connection to an Excel table. It's fairly easy to cut and paste from the selection guides into Excel, and Excel is fairly easy to maintain. The TS data really just needs to be a single table, anyway. Putting this data into a database is overkill, and you will probably end up regretting the decision if you go that route.
You can install PSD-Lite and see how the TS database is integrated with the box design tools. It's pretty nice, but as I noted before, the next version will look the same but connect to an Excel table rather than an sdf file. Notice that the program also includes a passive radiator database.
http://www.audiodevelopers.com/Software/PSD_Lite/setup.exe
Last edited:
I guess that came off as VERY discouraging. But the main point I was trying to make is that a "conventional" database is not going to generate much enthusiasm and will probably become a maintenance headache. And nobody is going to contribute. In fact, if you look at the PE Techtalk "stickies", there is a TS database thread that really hasn't been updated since 2013, and there are no contributions from anyone other than the OP. So even when PE provides a thread that everyone has to trip over, user participation is poor.
But I hadn't noticed that you were addressing the maintenance issue by the use of scrapers. That's a better approach, but that's been done before, also. There is a thread from 2011 on PE in which someone demonstrated a very good scraper and database, and it is still available on his site: Parts Express Power Search . The database hasn't been updated in years, but it's a good demo of how an open database with automated updates could be implemented.
Something like this might be a nice capability if some of the design tools could easily interface to it. For example, there could be a box modeling tool that allows users to access that online database, to quickly do a number of "what-if" designs on various drivers. But that's going to require getting a number of software developers to agree on the design, and that's not something that happens often in nature
. So I think you need to figure out how to get other developers to use the database before you jump in, and you should probably address the linkage to measurement data, since that will be a greater attraction for the design tool developers than just TS data. PE is now posting measurement files for their drivers, so there is an increasing amount of data available. But unless your database is linked to a loudspeaker design tool, it will probably get used as infrequently as the one at exstatic.org.I hope that helps.
Last edited:
I hear ya. Even with scraping, some sort of manual review of the intake would probably be helpful. Perhaps a review/unreviewed state, which would also facilitate manual data uploads from users (if that's a feature worth having). New driver comes out, user uploads info with a reference to a data source, app admin receives a message about the upload and reviews the data before flipping the reviewed switch. I've also made a note to create a process to allow people to report incorrect data.
There are definitely many potential issues with a project such as this, but as long as the pros outweigh the cons I'm willing to put the development time in.
I guess that came off as VERY discouraging. But the main point I was trying to make is that a "conventional" database is not going to generate much enthusiasm and will probably become a maintenance headache. And nobody is going to contribute. In fact, if you look at the PE Techtalk "stickies", there is a TS database thread that really hasn't been updated since 2013, and there are no contributions from anyone other than the OP. So even when PE provides a thread that everyone has to trip over, user participation is poor.
But I hadn't noticed that you were addressing the maintenance issue by the use of scrapers. That's a better approach, but that's been done before, also. There is a thread from 2011 on PE in which someone demonstrated a very good scraper and database, and it is still available on his site: Parts Express Power Search . The database hasn't been updated in years, but it's a good demo of how an open database with automated updates could be implemented.
Something like this might be a nice capability if some of the design tools could easily interface to it. For example, there could be a box modeling tool that allows users to access that online database, to quickly do a number of "what-if" designs on various drivers. But that's going to require getting a number of software developers to agree on the design, and that's not something that happens often in nature
I hope that helps.
Thanks for the feedback, Neil. You and Ron have been in this game wayyy longer than me, and it's great to get your perspectives.
Getting other developers on the bandwagon is definitely one of my goals here. There are a lot of disparate speakers resources scattered around the web, many of which are outdated or require a very specific operating system to function. Each resources that embeds its own database is ripe for inconsistencies with other similar resources.
I'd like to come at this from a microservices point of view. Box modeling and crossover design software both need driver data, so to me it makes sense to start there. With a driver database service that does one job and does it well, it can act as a foundation for other services via integration. I think this approach also aids in the viability of the project.
Good point on including the FRD and ZMA files in addition to standard parameters. Advanced search functionality coupled with having all three pieces (params, FRD and ZMA) in one place for each driver would be very valuable, imo.
I came across PSD-Lite last year when I first jumped in, but I haven't had a chance to play with it because I don't have a Windows installed anywhere, or Excel. I'm on a Mac at work and run Ubuntu at home, which is also part of my motivation for this undertaking.
With that said, what are some things that would make such a service valuable to you as the developer of PSD-Lite?
I'd have to think about this some more, but two things come to mind:
First, it would be nice to have a standard schema that everyone is comfortable with. The schema shown in one of the posts in the PE thread titled "Measured T/S Parameter Thread Sticky" would be fine with me. It includes fields for FRD filenames and some general-purpose annotation fields, which can be useful. It also has fields for "author", which is important for identifying the source of the data.
And second, the developers will need some examples on how to connect to the database and browse. Dave Ralph's WinPCD and my PSD-Lite both use .NET. Once we have some detailed examples on how to connect to and use the database we can implement the code and maybe get some more people using the data.
There are a couple of long threads on open source and cross platform tools at PE and I know that there were some other developers who might be interested. Try posting over there to see if you can stimulate some interest.
I'd be happy to give away the code I have for box and baffle design, and make it open source. The code includes an FRD/ZMA file editor with a minimum phase extraction routine. But my code started many years ago, and it uses VB.net and Windows Forms. In fact, it started with .NET 3.5, which didn't have support for complex numbers in VB, so my code uses a complex number library that I had to write. So I don't think anyone really wants my code
I'd have to think about this some more, but two things come to mind:
First, it would be nice to have a standard schema that everyone is comfortable with. The schema shown in one of the posts in the PE thread titled "Measured T/S Parameter Thread Sticky" would be fine with me. It includes fields for FRD filenames and some general-purpose annotation fields, which can be useful. It also has fields for "author", which is important for identifying the source of the data.
And second, the developers will need some examples on how to connect to the database and browse. Dave Ralph's WinPCD and my PSD-Lite both use .NET. Once we have some detailed examples on how to connect to and use the database we can implement the code and maybe get some more people using the data.
There are a couple of long threads on open source and cross platform tools at PE and I know that there were some other developers who might be interested. Try posting over there to see if you can stimulate some interest.
I'd be happy to give away the code I have for box and baffle design, and make it open source. The code includes an FRD/ZMA file editor with a minimum phase extraction routine. But my code started many years ago, and it uses VB.net and Windows Forms. In fact, it started with .NET 3.5, which didn't have support for complex numbers in VB, so my code uses a complex number library that I had to write. So I don't think anyone really wants my code
Thanks for the guidance, and I'll start reading the "Measured T/S Parameter Thread Sticky" thread soon enough. As for open sourcing your work, let's chat about that once a database is up and running
I'm going on vacation soon, but I'll update this thread once I have made some progress with planning.I took an initial stab at an ERD for phase I of this project. All feedback is definitely welcome, and I'll soon create a similar thread at PE to try and maximize feedback before anything gets implemented.
An externally hosted image should be here but it was not working when we last tested it.
In the picture are the parameters I use...
This is the minimum set useful, IMO. Many drivers do not come with Qes, Qms values and a method to enter these drivers would be useful. I just used Qts and Qes, but you could use Qts and Qms - fairly simple to just guess a value for Qes or Qms and it doesn't affect calculations a huge amount, as Qts is the most important parameter.
ignore pdissipation as that depends on a user input for power.
This is the minimum set useful, IMO. Many drivers do not come with Qes, Qms values and a method to enter these drivers would be useful. I just used Qts and Qes, but you could use Qts and Qms - fairly simple to just guess a value for Qes or Qms and it doesn't affect calculations a huge amount, as Qts is the most important parameter.
ignore pdissipation as that depends on a user input for power.
Attachments
Attached is the updated ERD. I've expanded the fields contained in SpecSet considerably, and also added a Material model.
What do the following from your attachment represent?
- Pe_max
- Mdt
Attachments
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.