Have some basic knowledge in ml

>have some basic knowledge in ml
>apply knowledge and build an algo to trade stocks
>beat the market consistently
it can't be this easy can it? Why isn't everyone doing this?

Other urls found in this thread:

github.com/Jamonek/Robinhood
quantiacs.com/For-Quants/GetStarted/QuantiacsToolbox.aspx
course.fast.ai/
bookzz.org/book/816657/eaf623
deeplearningbook.org/
see.stanford.edu/Course/CS229/
quantquote.com/purchase.php
quantopian.com/posts/sample-algo-vwap-with-variable-short-slash-long-volumes-1
youtube.com/watch?v=tNXo8cu-hmo
blog.shakirm.com/2015/01/a-statistical-view-of-deep-learning-i-recursive-glms/
machinelearningmastery.com/tutorial-first-neural-network-python-keras/
ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-034-artificial-intelligence-fall-2010/index.htm
bookzz.org/book/1267941/f21e8e
twitter.com/NSFWRedditImage

Trying to get out of silver hell in CS:Go.

you're selling the algo then or using it?

using it, might sell it when it stops working

I could never get past silver3. Just quit bro and find something more productive to do

Proof pls larpfag

Here you go desu

what are you invested in?

reeeeeeported

I've taken a graduate ML course at my college, what did you build this system in and what techniques did you apply?

nothing right now. Will know tomorrow what meme it picks.

Share source code plz.

How did you hook it to robin hood or is it a signal generator? Do you do daytrades or longer term being under 25k? How awesome was it when it actually worked?

I am building mine right now and think I will have it generate signals for me to follow until I get to 25k. What language did you use? Did you build it from scratch? Mine is a concurrent neural net.

*convoluted not concurrent

Might not perform too well once it has more capital though who knows. Still even if you just let it make you a couple grand a month and it stays consistent that's more than some make in a year.

It's convolutional you fucking idiot

Whatever, doesn't really matter as I developed mine independently and it does not really match any out there ATM. Convolutional shares some structural features so I use that to describe it.

Really though I just want to hear ops story, not argue about networks.

Friendly reminder to charge your phone and update the SuperSU binary!

Any resources you can point us to? At one point I was starting to get into the tensor flow stuff, but got distracted with other things. Do you need to know a lot of math or do you just abstract that out (tensor flow seemed to really push that idea).

ML is the future. Do well in that course

>what did you build this system in and what techniques did you apply?
built in python with numpy/scipy. Scipy to help root find a global minima and numpy for performance. Uses robinhood's api.

One day desu

>How did you hook it to robin hood or is it a signal generator?
github.com/Jamonek/Robinhood

>Do you do daytrades or longer term being under 25k?
It just does daytrades, I'm planning on dumping more money in it once I get a job so it can daytrade without the PDT limit (more gains).

>What language did you use? Did you build it from scratch?
Python, and yes

>How awesome was it when it actually worked?
I was pretty elated. Treated myself to a chocolate milkshake. It's a great feeling tbqh. I dread the day when it doesn't work though.

Convolutional neural nets are really powerful. I'm considering creating one next, but I need to study up on some literature before I do.
What helped me a lot was risk management in my code. (Trailing stops, risk tolerance etc)
Make sure you backtest your strat and pay close attention to periods of large drawdowns and why they exist. Also avoid overfitting. Check out quantopian.com for backtesting. Good luck and see you on the moon

Yeah, I have no idea right now since the capital is pretty small. I'll shut it down if the drawdown exceeds 40%

I'm majoring in ML actually, it's sort of a combination comp sci and statistics major here. I have some experience with Python so I'll give this a look. How complex is your system? How long did you spend writing it?

Care to share a few tips? Like what kind of data you're feeding it or which ML algo you're using?

Can you prove that's active trading? I made about 60% this month from a couple of speculative holds.

>look at my great algo
>mutilated graph forced onto small smartphone screen
okay, very professional

Are you retarded?

Do you think you need to work at a hedge fund to write a trading algo?
Anyone with an IQ over 100 could do it desu.

A robin hood api! Thank you a thousand thanks for that.

>Make sure you backtest your strat and pay close attention to periods of large drawdowns and why they exist. Also avoid overfitting. Check out quantopian.com for backtesting. Good luck and see you on the moon

quantiacs.com/For-Quants/GetStarted/QuantiacsToolbox.aspx

That will save some time, and they have market history.

Thank you again good sir, I will follow your advice and test the algo ruthlessly. Does the quantiacs toolbox allow for WFA? If not I will build a system to do that as well. See you on the moon good sir.

Wew im impressed

Until you've run a backtest and know the algorithm's alpha and beta, it's not really shit.

How much confidence do you think one have in the algo given that it has back tested well and isn't overfitted?

Yours is better?

lol thanks man, need to replace the battery. Old lg g2.
I have never played with tensorflow so I can't say, but I hear it's good since everyone talks about it.
Check out course.fast.ai/ and the coursera course for machine learning.
You don't need to know that much maths, having sufficient knowledge of Calc2/3 and some linear algebra will suffice if you really want to understand certain concepts.

Math helps in understanding how things work, but in the end results are all that matter. Just learning and playing around with tensorflow would get you pretty far I think.

Good major choice. Kind of wish I didn't take CS and majored in math. The code isn't complex at all. About 230 loc. 100 lines for the ml part and another 130 for risk management/placing trades. I spent probably about 6 hours initially writing the ml part and the other 6-8 hours for everything else. I've spent more time taking it offline and tweaking it than I spent writing it.

>Care to share a few tips? Like what kind of data you're feeding it or which ML algo you're using?
Buy low, sell high mah nigga. Gratz on the 60%. Retail trading isn't for me. I have a bit of a gambling problem

I haven't tried quantiacs, but it looks really interesting. Thanks for the link and no problem dude. Glad I could help a fellow algobro.

good point

What's some good entry-level stuff on both the finance and statistics part of this? Knowing what to analyze, and how to do it.

Math major here. What courses are relevant for ML?

Wait nvm forgot how to read.

Any advanced courses that would be good to take? Stochastic processes, PDEs, Graph Theory, Operations Research, maybe?

could a 2nd year computer engineering student pull this off? I want something to work on in my downtime

If you are determined then it is a matter of when not if. If you have a background in computers then it will be a matter of R&D, luckily all the resources are out there.

Here is a book on developing and testing algos. Very important to learn, like said it is important to test your algo. Ml is notorious for curve fitting.

bookzz.org/book/816657/eaf623

For me learning about trading with algos was the biggest challenge, still is. The ml and programming is a matter of time, however, stock market algos are rather new to me. Such as the risk management he mentioned.

Book is The Evaluation and Optimization of Trading Strategies by Robert Pardo. Second ed.

cheers for this, much appreciated.

No problem my friend.

Noob at programming here, but I have background in statistics, namely SPSS. could I make a shoddy algorithm using SPSS and some beginner Java? Java seems to be the easiest to learn

Yes, programming is literally just breaking things into individual steps so the computer does what you want. Like telling an old person how to use the computer, move mouse here, click, move mouse over here, now click. Each step explained for something like clicking save in word. The syntax might be a bit confusing but and semi intelligent person can manage.

I'm a professional software dev but a bit of a layman when it comes to ML. However, I thought in order to do deep learning you needed to train your algorithms on a large dataset. If that's the case, how did OP acquire this dataset? Wouldn't you need a large ledger of past market transactions? Or did you simply gather that for a few days yourself and then let the algo do it's work?

This is by far the most useful thread on Veeky Forums btw, It's motivated me to try my hand at learning all this and applying it.

Here are some resources for learning ML/DL that I've scoured that seem very reputable.

A seven week hands on course
course.fast.ai/
A book that goes deeper into the necessary mathematics and tradeoffs of various DL techniques
deeplearningbook.org/
A online course on the field of Machine Learning taught by one of the foremost experts in the field
see.stanford.edu/Course/CS229/
This book specifically on trading strategies posted in this thread
The Evaluation and Optimization of Trading Strategies by Robert Pardo. Second ed.

May not need all of this but it covers all the bases I think. I am going to do them in order I posted.

What are you training it on? I'm interested in doing sentiment analysis on news to determine stock picks, is this the right way to go? Surely you're not beating the market from just training on raw price data.

This seems like a good thread to post this
Could one monetize a web crawler?

If so, how?

Any sources on building crawlers?

I like how you cant even use google but think you can get rich with web crawlers

Quant toolbox contains a history set for training and backtesting.

Markets
Quantiacs provides data for the Stocks in the S&P 500 and 44 US Futures Markets. Any combination of these are selectable for your strategies.

Doesn't say how far back but opening the file will probably tell you. Might not be the best source but it is there.

IIRC in an old algo thread they were talking that there is a place that sells market data for every stock that was, not just the ones alive today. Costs $3000 and they send you hard drives with terabytes of data. Using only surviving stocks will give your algo a skewed training set. If anyone has more on this it would be appreciated.

Skimming through the futures in the toolbox some go back to 1990. I would dig more but time for work.

>Buy it, reformat those hard-drives, sell for more than $3000 + shipping

be /biz

Only has the DATE OPEN HIGH LOW CLOSE VOLUME OI P R RINFO so its pretty limited. Will not be able to do day trading on that one.

Selling wiped HD would be a Veeky Forums kinda deal.

quantquote.com/purchase.php

$9000 for the complete set of all symbols. Best resolution is 1minute intervals. Should be suitable for training data, if it means what I think it does. Can select less things and get a better price, 9k is the whole thing.

Thank you, I'll check out that course and wiki they provide.

Do you have any advice on where you found / got knowledge on the financial aspect?

Anyway I found this a project call zenbot, which is a ml based BTC trading bot somebody open sourced. While it's technically a bad idea to reuse someone elses alg. I may pick it apart a bit and give it a whirl with a few dollars.
What's really interesting about zenbot is apparently it doesn't take price into consideration only buy and sell volume. Though I haven't looked at the code or even really know how to analyze ml code.

>beat the market consistently

Stupidity like this will result only in your perpetual confinement to your parent's basement.

.Dream on, kiddo

That's the Robinhood UI you fucking idiot

I cannot imagine having this defeatist attitude, would be a living hell imo.

Stochastic processes, graph theory will be useful. I don't think you'll need PDE . I'd say since your a math major, having good knowledge on algorithms/datastructures will be good enough for ml other than taking ml courses.

heaps of info online. Investopedia for finance knowledge and a basic college level course for stats.

sentiment analysis is pretty underrated. Hard part is backtesting. Check out the stocktwits api

check out scrapy/beautifulsoup for web scraping. Play around with some sample programs

most of the knowledge I gained from reading up on things in Investopedia I found interesting while browsing Elitetrader for fun. Elitetrader has heaps information.
Also nothing wrong with reusing code, you don't want to reinvent the wheel, just make sure you understand the code you reuse. I took a look at zenbot. It seems to be using zenbrain for the ml part. You could probably figure out the ml by reading up on the zenbrain api and seeing how zenbot uses it.

Silver hell?

Nah. If it was that easy everyone would do it like u said. How many people honestly answer no when u try to hand them 10 billion dollars. Very very few. Wall street has deep pockets and supercomputers trading hundredths of a cent changes.

Stocks are for cucks anyway. You get in pre ipo as an accredited investor or if you have too much cash on hand and have a gut feeling you play options and NEVER gamble more than 4% of your capitol no matter what. No idea why but i have heard that shilled a lot.

Odds are u wont lose 20 times ina row i guess but it could happen in a crash. At the same time real options traders know that while sad. A crash is good for them. A good options guy can make money at the moon at a crash and fuckin sideways. They just happy it moves when headed up or down.

Sideways stagnates but u can still "rent" some stocks. Have old dependable shit like general electric or berkshire Hathaway. Never really moves but u have it. Whore it out for the other options traders and then win lose or draw you are good. One way you get paid more than u figure the stock is worth and sell it or u get it back and not sure if u make any money there but u didnt lose anything and had a chamce to make bux with little effort.

DOW is at an all time high you psycho fuck. If not larping cash out and SPRINT TO GOLD AND SILVER FUCKING NOW. Not kidding, samefagging, shilling, or trolling. Honest to fuck everyone screenshot this fuck and report back after the crash to congratulate OP or laugh at his broke cuck ass and get him a hard lesson lernt. Not fucking kidding. At all... broke af Kansas fag with a big block limo and square body daily driven for future timestamp and shitposting. Assuming this cocksucking piece of shit fucktard ETF cunt doesn't have all 200 threads up and bumped anything fun or useful away with his samefagging shit. Honestley never wanted to cut a stranger's half inch dick off and make him eat it while getting a giant KEK tatoo on his eyeballs so fucking bad in my life...

you're the psycho one

I'm sure your "basic knowledge in ml" is going to beat all those Harvard and MIT math PhDs with giant supercomputer clusters ;^)

I need tick data on meme rockets

You can get live data from yahoo iirc. You have to pay for the data I linked. Doesn't appear that you can get good data sets for free of old data.

How much would you reasonably sell this thing for?

What makes you think he had to beat them? That is the same as saying no one can trade stocks because you cannot compete with the big firms.

Not to mention that you need a quite powerful machine for proper DL. I'm talking multiple GPUs.

Non-trivial use of DL is for the big boys pretty much.

AWS, Google Cloud, Azure.

Pick one and get to work and stop making excuses.

>I took a look at zenbot. It seems to be using zenbrain for the ml part. You could probably figure out the ml by reading up on the zenbrain api and seeing how zenbot uses it.

Yeah they are written by the same person. Not sure if I'm retarded, but it doesn't even appear to be machine learning. Zenbrain is all map and reduce functions. As far as I can tell it just pulls data from your preferred exchange, then you manually tweak the algorithm and run the simulation.

bump

>About 230 loc. 100 lines for the ml part and another 130 for risk management/placing trades.
Well back to basics for me, all I can find are these crazy convoluted examples with thousands of lines of code and no clear way to pipe historical data in.

Ever since they added skins csgo filled up with 15 year olds who think they are hot shit. Find a new game. It isn't worth it

yeah it applies weights to events. So more recent events are weighted more and have a higher impact on any decision. That's the ml part of it from what I see.

quantopian.com makes it super easy to pipe historical data and backtest your strategy. A simple VWAP program is about 40 lines of code. Even if you can make 10% a month, you can leverage your strat with options which is what I plan on doing eventually. quantopian.com/posts/sample-algo-vwap-with-variable-short-slash-long-volumes-1

I could probably sell it as a black box scanner on a SaaS business model, but.. I feel like that's too much work and the liabilities as well if something goes wrong.

Because those are accounting for 95% of the trade volume.

So? How does having a bunch of algos trading mean that new ones can't?

Python will be easier for web scraping. Also probably an easier language to learn and quickly get some code running with

youtube.com/watch?v=tNXo8cu-hmo

wtf, has anyone tried this?

>ironically posts with mindset that will doom him to his parents basement

Fuckoff queer and go take your meds u mental fucking autist

/thread

What these threads always boil down to is:

> "guys I can beat the market!"
> How long have you been trading?
> "...under 6 months"

You are not helping your case.

I'm just trying to motivate you guyz. Why are you being so mean?

bumpity bump

Others may be interested in the python library: keras
It's like tensor flow on steroids

OP how did you choose what type of layers to use? Like softmax vs rectifier vs sigmoid. The above (keras) is really cool, but it opens the doors for everything and besides experimentation it's not really clear where one would be used over another for particular use cases.

Cause you're fake bro.

Come back in 2 years when you have turned a few bucks into millions.

Which you will surely do if you have created a model which can beat the market.

But you won't because you haven't.

You think you are the first trader to get some beginners luck and rush to tell Veeky Forums that they know how to beat the market?

No, it's legit not that easy sadly.

You still need some serious capital to begin with, and decent infrastructure too. There's a reason that HFT firms are often located, or have DC equipment co-located, near to exchanges.

These type of threads always devolve into the same shit flinging. Everyone comes out of the woodworks to say why it's going to fail and generally be nasty about it.
It's really not that hard to believe the OPs claims. Legitimate criticism would be the short duration of run time, a few months isn't enough to say it will be profitable forever. And op's alg running into problems at scale if he were to throw millions of dollars at it (and not accounting for the fact that it actually has an effect on the market it's trying to predict).

>hurr you can't be the very best so don't bother trying

keras is the shit dude, best part is you can use it with a gpu. You will generally end up using sigmoid/tanh for the most part in your hidden layer activation and output layer activation. Softmax is used for your output activation in a multiclass classification neural net. I don't know about relu other than that they are used in sparse nets and are really good for that specific case.

This site does a good job of showing what activation functions are used for which problems.
blog.shakirm.com/2015/01/a-statistical-view-of-deep-learning-i-recursive-glms/

>implying this is beginners luck
Live trading results are lining up with backtest results. The strat might stop working who knows, but this isn't beginners luck desu

You can trade on a larger timescale and avoid competing with HFTs completely

Unfortunately I know it won't scale :/

Not sure why you replied to me there, I wasn't "shit flinging" - just saying that it's "legit not that easy sadly".

No hyperbole, no accusations, and no nastiness.

The main failure in my post that you could've picked on was actually my focus on HFT.. ;)

Apologies for focusing on HFT, dude.

To be quite honest, I haven't even read the full thread - I just see quite a lot of people who think ML/AlgoTrading is a quick path to instant riches; these people tend to focus on HFT.

To be fair to you, it seems like you're grounded and aware of any shortcomings, plus you've also done your research about the technicalities - that puts you ahead of a lot of people.

Keep at it: don't count your chickens before the hatch obviously, but ultimately it's some good experience and you don't gain anything if you don't try.

Good luck, bro!

Again thanks a bunch for the info

>my focus on HFT..
This is exactly what I meant, OP says he did one thing and everyone jumps on him saying he can't achieve the most extreme end of the spectrum possible.
You weren't nasty about it though, so fair enough. All the defeatist attitudes in here just got to me.

It is actually a valid and interesting point to bring up for algo trading. I find it fascinating what some regular dude like op can do as well as what little information the public knows about quants. They also pay big money for microwave links to the major cities too - less bandwidth but more importantly much less latency than even fiber.

You motivated several people from the looks of it. I say mission success.

Decent programmer here. Even though I have been programming for more than 10 years, ML is very intimidating and it seems to require a lot more more statistics and math than what I'm used to dealing with. Even "basic" tutorials and books I have tried throw you right into it and it feels like I'm following the code but not the process.

My question is: How do I ease into it do I go and improve my math first and then come back or does it get any easier and starts making sense at some point if you just grind ML material?.

>My question is: How do I ease into it do I go and improve my math first and then come back or does it get any easier and starts making sense at some point if you just grind ML material?.

I'm in the same boat so I don't know, but my 2 bits is that I'm a half decent programmer, not so great with math and algs though. I found the keras library to be REALLY nice at abstracting out just about every little detail. There's a ton of tutorials showing how to use it and my only complaint would be they don't actually get into the math/how/why you build a network a particular way (op has given some good info in this regard).
Once I'm comfortable with that I may go back and use just tensor flow (keras rides on top of tensor flow), and then maybe go and try in C or something.
Like cryptography there's the theoretical algorithm and then there's the real life implementation which has many subtle nuances. So jumping straight into the math isn't something that would work for me, and really is less important. You don't need to fully understand AES to encrypt something.

This guy puts out a lot of good stuff:
>machinelearningmastery.com/tutorial-first-neural-network-python-keras/
bonus points that it shows how to pull in from a csv file and not some mysterious python "import mnist.data"

On a different note:
>tfw digit recognition at >98% success on test data
>feed in my own image
>mistakes a '1' for a '3'
>feels bad man

>mfw after 100 hours in CSGO in the span of a few weeks I was MGE
Get fuukkin gud

Anyone here have any info for selecting features of a stock to focus on for your ML? Maybe OP?

You just have to try until something sticks. Volume, VWAP, technical indicators, rsi, oversold, overbought etc. On a side note, you can focus on a specific market which is much easier and build a model for that. Example: predicting spot price of natural gas based on weather patterns, the weekly draw reports, futures contracts, heating and cooling degree days etc. If you can build a algo that's even 60% right, you can play the DGAZ/UGAZ etfs. Even leverage that further with options if you're confident.

not all of us are born git gud mate

what said, just understand the concepts behind them. Don't get too worried about the math, and study it side by side. Calc2/Calc3 and linear algebra is more than enough in my opinion to get a good grasp of the math behind most ML techniques such as neural nets. Some machine learning algorithms require no math or very limited amounts of math such as genetic algorithms.
ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-034-artificial-intelligence-fall-2010/index.htm
This is the course I followed. The videos are on youtube and pretty fun to watch/learn.

Thanks bro, that mean's alot.

I hope so desu

here, not sure if ID changed. Any specific models you used? I just started a course, just learned Naive Bayes and moving into SVM next. Thanks for all the help!!

np! I use regular feedforward backpropagated neural nets and genetic algos

newfag here, how can i start to trade? what am i supposed to trade? how can i buy stocks. please help me

>oh, you're not going to make infinite money so your means of making passive income on a consistent basis is shit

scalability is meme. yeah, he's not gonna make trillions, boo hoo. This is fucking reality and even making enough to make ends meet is worthwhile.

This, breaking out of wageslave status would be a success to me. I know I will not be making millions, but if I can make enough to spend time working on my algorithms I can improve more and more.

you can open an account with robinhood. Invest $50/100 and get your feet wet. It's commission free so you won't get raped by fees.

There are different styles of trading such as swing trading, day trading, shorting, momentum trading or just buy and hold. Important thing is not to gamble by letting your emotions run wild! And stick to a set of rules.
One book I would definitely recommend you read is Trading in the Zone by Mark Douglas.

There is a robinhood general thread on Veeky Forums, but I wouldn't recommend it as it's full of pennystock shilling. You might learn a thing or two though on there though. Best bet is to study up on fundamental and technical analysis.

Trading in the Zone by Mark Douglas

bookzz.org/book/1267941/f21e8e

Can you open source your algo/program? It would help a lot of us get started. Maybe more experienced Veeky Forumsmen could contribute and we make it the official Veeky Forums trading algo?

What do you think guys?