Looking for a Database Programmer

[tillyoubreakit]

I think i wrote that on the first page. I do not want to store the videos and audio files in a databse. Server 1 with database contains the links to the content on a fileserver.

We dont want to replace google (thats not possible), but more to make a technical library. I think we will start with Modell Airplanes.

You guys think to BIG , what i was laying out is the optimum in the future. Right know all we want to do is, to create a small cute little base which is modular and can be expended at anytime as the size of the content groes. And if this is to become a success than i am sure we are willing to spend an awfull amount of cash on it. But we just start with a small cute basis and if it works then will invest in it.

And hey if it doesnt work will try the next idea, it aint hard to make huge amount of cash in this didgital life ;-)

[Stoo]

Now this is where the confusion is creeping in..

You've got a thread asking for a "Database Programmer", then talk about a system that can "Hold min 1TB of 'Text' 'Video' 'Pictures' ", and "Can have each selection from the above collected and displayed in a webbased formular in less than 5 seconds".

It might seem small, but it'll soon get *very* complicated, then you've got the headache of bandwidths and how you're going to deliver the video files (how big are the files? customer download? streaming? what sort of streaming? is it viable to transfer that amount of data to customers over the internet?).

[tillyoubreakit]

I know it its confusing, i caused a lot of it.

We need something thats small but can be scaled to super large once the thing starts rolling. We dont want to Stream anything. You can have the Videos and Audios either by CD or by downloading the file as a whole. The files should be around 50 or so MB, nothing to big. Will just burn the big ones on a CD.

Well my second meeting with the other person is tomorrow. Will clarify much more and than i hope i can you give you all a much more detailed version of this.

-Florian


PS: Link to a sorta Hardware modell http://tillyoubreakit.is-a-geek.com/Drawing13.htm

[dgr]

PS: If you still go to school, and wouldnt take a JOB <--better than no job and living off parents and are willing to laugh at 8000lbs than you must be pretty rich already.:-)

3. I was not thinking of paying a minor 8000lbs in britain. It seemed to me like he was saying "I am too good for this, and i laugh at 8k lbs".

i am in fact a university student. i'm not particularly rich, but the reasons i made my points were:

*1TB of data is a hell of a lot. thats at least 10 drives, probably more, probably involves redundant PSUs, (hefty ones at that). you're talking quite a bit of kit. That means quite a bit to administer...

*The programmer is going to have to be pretty damn good at database programming - rollbacks, ACID, transactions, eaking out performance (because you're going to have to doing so to make this system even acceptably responsive)...

*In addition, he has got to figure out a way of presenting this data - so he has got to know .NET, python, whatever.

*You're going to have to figure out a maintainace contract...

*And all this seems to be paid for some system of micropayments - that you haven't necessarily even figured out yet!

I don't mean this as a flame / troll, but its just my oppinion that this is at least a two man job - and that the requirements haven't been thought through well enough for me to be interested.

Just my 2c.

[dgr]

What you are asking to be done isn't the work of a programmer, this is the work of a decent project manager

i agree 100%

A rough estimate of the cost of this to setup from my experience would be in the several million mark to do properly and get a system that maintains itself. Sure everyone might like £8k for offering a solution, but quite frankly i'd prefer a £100K slice if this project was truely an option.

TiG

that might (IMO - you could be completely correct) be OTT, but still nearer the mark!

[Moby-Dick]

*1TB of data is a hell of a lot. thats at least 10 drives, probably more, probably involves redundant PSUs, (hefty ones at that). you're talking quite a bit of kit. That means quite a bit to administer...

1TB of Data - you could do as 9 140Gb SCSI drives in a RAID 5 Config.

Most Disk Shelves ( for DAS ) would have a redundant PSU - they are designed to run 14 or so drives per shelf so you wont be needing to uprate.

If the server is in a decent Data centre the they'd be on a building UPS , so it shouldn't be a problem.

Bandwidth charges would be nasty though ( but doable , providing your business model is good )

From a systems point of view , a quad opteron box with plenty of memory would easily handle it.

If its business critical , then you want to think about clustering ?

infact an On-Demand solution could be ideal as the backend is instantly scaleable.

I'm a systems guy , not a db person , so thats all the advice I can give

If you didn't want to go SCSI for storage , you could go SATA , but build in a much larger fault tolerance factor.

[Stoo]

Quick and dirty priceup for the storage server:

Description Quantity Total excl.VAT Total incl.VAT
APRIL SPECIAL
Dell PowerEdge 4600
£8,290

£9,740.75
Standard Features: ServerWorks Grand Champion High End chipset
Provision for the lastest Intel DP Xeon processors
Up to 12GB of DDR SDRAM memory DIMMS
Available as either a tower or 6U rack-optimised chassis (7 servers per 42U rack)
Optional embedded PERC 3/Di Dual Channel RAID controller with 128MB battery backed cache
Integrated Intel PRO/100+ and Broadcom Gigabit NIC
Seven PCI expansion slots: Six 64bit/100MHz PCI-X, one 32bit/33MHz PCI
Includes 2 Serial ports
Embedded systems management for monitoring critical components, active ID,
LCD panel for status checking, and status LED's for critical components
Storage Expansion for up to eight 1'' hot-plug Ultra3 160MB/s SCSI hard drives and a further two 1'' drives in an optional media bay (10 total)
Dual built-in Ultra160 SCSI controllers (for hard drives)
One Ultra/Narrow SCSI (for CD-ROM and tape drives)
Hot-Plug Redundant Cooling Fans and Power Supplies with individual Power Cords. Redundant power supply option.
Supplied with CD-ROM drive, Performance Keyboard, Microsoft Mouse & 3.5'' Floppy Drive
Two USB ports
Certified for clustering support.
The Dell OpenManage suite to enable automated/rapid operating system installation, and complete network management with IT Assistant.
30-day Getting Started Support for new installations
Purchase your preferred Operating System from Dell and we will factory install it free of charge.
1024MB 333MHz DDR RAM (2x512)
Maintenance Service: 3 Years Gold Service (Direct access to Dell experts 24x7)
Business Installation: No installation
Special Requirement Installation Service: Please contact us at Dell to discuss your specific requirements
Chassis Orientation Options: Tower Chassis
Rack Mount Kits: Not Included
Processor(s)/Cache: Dual Intel® Xeon™ processor 2.6GHz with 512KB L2 cache (+ GBP £450)
Hot-Plug Power Supplies: Three+one 300W redundant Power Supplies (+ GBP £73)
Memory: 2GB DDR SDRAM (4x512MB 200MHz DIMMs)
Advanced Memory Protection: Standard memory protection (ECC and Chip-kill support)
Keyboard: Dell Performance Midnight Grey Keyboard
Monitor: Dell 15'' Value Flat Panel Monitor (+ GBP £270)
Main Bay Hard Drive Backplane: 1x 8 hard hot plug hard drive backplane
1st Internal Hard Drive: 146GB 10,000rpm 1'' U320 SCSI hard drive (+ GBP £295)
2nd Internal Hard Drive: 146GB 10,000rpm 1'' U320 SCSI hard drive (+ GBP £435)
3rd Internal Hard Drive: 146GB 10,000rpm 1'' U320 SCSI hard drive (+ GBP £435)
4th Internal Hard Drive: 146GB 10,000rpm 1'' U320 SCSI hard drive (+ GBP £435)
5th Internal Hard Drive: 146GB 10,000rpm 1'' U320 SCSI hard drive (+ GBP £435)
6th Internal Hard Drive: 146GB 10,000rpm 1'' U320 SCSI hard drive (+ GBP £435)
7th Internal Hard Drive: 146GB 10,000rpm 1'' U320 SCSI hard drive (+ GBP £435)
8th Internal Hard Drive: 146GB 10,000rpm 1'' U320 SCSI hard drive (+ GBP £435)
Internal Hard Drive RAID Configuration - SIMPLE: C12 RAID 5 using add-in RAID controlle- requires three or more hard drives of equal size/type
Internal Hard Drive RAID Configuration – COMPLEX: C5 RAID 5 for internal drives (embedded RAID) with hard drives in media bay connected to SCSI controller
1st Controller Card: PERC4/DC U320 RAID Controller (128MB cache) (channels: 2xINT, 0xEXT) (+ GBP £597)
2nd Controller Card: Not Included
Storage Device and Recordable Media: 24X Max IDE CD-ROM Drive
Internal Tape Backup or Hard Drives (Media Bay): Not Included
Internal PowerVault Tape Backup Drives (Media Bay): Not Included
External PowerVault Tape Backup Drive: Not Included
Tape Backup Essentials for PowerVault Tape Drives: Not Included
Tape Backup Software: Not Included
Premier Enterprise Software Support: Not Included
Uninterruptible Power Supplies: APC Smart-UPS 700i, 450W (+ GBP £173)
Communications: Not Included
PowerConnect Network Switch: Not Included
OpenManage Subscription Service: No OpenManage Subscription
Factory Installed Operating System: Not Included
Asset Tagging Service: Not Included
Software & Storage Support: Not Included

Just getting the storage up to a mission critical level on one machine blows the entire budget, and that's still missing all the backup options (not sure about the raid setup either, dell's system is a bit confusing ), OS, etc etc..

[Moby-Dick]

we just bought one of those 4600's for our new SBS box

4 x 36's and 3 x 73's for storage

[Moby-Dick]

also dont forget backups - RAID is not a backup medium.

you'll need a silly ammount of backup tapes

[Stoo]

yup, but by that point I'd had enough of dell's page

[dgr]

1TB of Data - you could do as 9 140Gb SCSI drives in a RAID 5 Config.

my figure of 10 drives was based on this: (8+1)*140gb for storage plus one hot swap ready for replacing a dead drive.

dgr

[tillyoubreakit]

Well getting 1TB of space is absolutly no problem. And the administration is no problem either. The 15k are just for the software (base).

The server will have of course a dual psu and scsi, anything else is a joke.

My software team will finish our current project and then take the cash from that to finance this. So money wont matter later on at all.

We figured that we dont have the power right now to run 2 projects at once, so this is delayed till the one right now is done which should be 3-6 month.

And hey if it all works out, i wont work ever again for the rest of my life and spend my time listening to a MG 20.1 on Bermuda......hehe will see. ;-)

I thank all for their input, and apologize that there were some confusions in this. But writing it all down, and explaining all the facts is very hard since most of this stuff changes in minutes.

-Flo

PS: If 24hr service aint important you can save thousands when building it itself and we get discounts too since we are a distributor ;-)

And estland provides 26MB line for free to bizz customers and 0% income tax. hehehe will see how it all turns out

[tillyoubreakit]

*1TB of data is a hell of a lot. thats at least 10 drives, probably more, probably involves redundant PSUs, (hefty ones at that). you're talking quite a bit of kit. That means quite a bit to administer...

The size dont matter, if i administrate 10 20 or 100 drives wont make a diff. And if one fails the system still runs.

*The programmer is going to have to be pretty damn good at database programming - rollbacks, ACID, transactions, eaking out performance (because you're going to have to doing so to make this system even acceptably responsive)...

Of course. but since ill take my team after the current project finishes its all good.

*In addition, he has got to figure out a way of presenting this data - so he has got to know .NET, python, whatever.

.NET

*You're going to have to figure out a maintainace contract...

No need for that, its all in my company

*And all this seems to be paid for some system of micropayments - that you haven't necessarily even figured out yet!

Once you get the masses you win ;-)

I don't mean this as a flame / troll, but its just my oppinion that this is at least a two man job - and that the requirements haven't been thought through well enough for me to be interested.

I dont wanna be mean, but your wouldnt be on my list anyways. Too young, still in school, and not enough expirience.

But i thank you for your interest and replys :-)

[Moby-Dick]

good luck

and I'll get a quick pimp while I can

if you would be intrested in an on-demand service (using IBM's E-BOD technology ) than drop me a PM

[scottyman]

Well, looks like I haven't quite missed the window on this.
I've been there, done that - and had my designer knock up the t-shirts.

1TB of Video and Images - not difficult, cluster it for performance/availability - you take a large latency hit.
DB - not too hard, integrity checking is tough, especially when you're talking large quantities.
My main question is - why on earth .NET (or if you're talking cross-platform which WILL be a requirement - .NOT)
Php works quite nicely, and of course Perl is an even better option if you're talking huge amounts of raw text data - it's just so much more efficient, and not to mention industry proven.

Rule of thumb - video between 1.5 (MPEG1) and 3.5 mb/s (MPEG2) allows up to 30 network users per 100mb segment.
Clustered Darwin/WM server for video distribution
Clustered Webserver for front end/image distribution (or chuch the images into the DB as blobs - but you take a latency hit)
Cluster MySQL db - then you're talking.

I did a larger scale job, but with java front end and video only - you were talking a good 30 seconds to precache video header information to ease loading times - bought video load times down to sub 2 seconds, images as they were stored as blobs took a latency hit of 5-7 seconds, and they were simply keyframes.

For a commerical implementation - I'd be looking at 80 grand off the shelf for software, or around 150-200 for bespoke, depending on level of technical expertise on the part of the supplier and client. - i.e. did client want to do it in house, then you're talking more dosh to bring in good system engineers to clean up the client's messes.

Even then, you're not talking a huge project.

[dgr]

My main question is - why on earth .NET (or if you're talking cross-platform which WILL be a requirement - .NOT)
Php works quite nicely, and of course Perl is an even better option if you're talking huge amounts of raw text data - it's just so much more efficient, and not to mention industry proven.

Without an OOP language, i don't know if anything like this could ever be maintained. neither php nor perl scales AS WELL as other langauges (though i realise that it is feasible) - my preferance would either be the .NET route, or a combination of python/ruby...

oh and:

I dont wanna be mean, but your wouldnt be on my list anyways. Too young, still in school, and not enough expirience.

No offense taken, even though i'm actually in university, i still don't think it would be worth it