Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
I'm certain I answered every question on HFT. It just turned out that there were more of other questions, which I felt was a good thing. If you're displeased, let me know which replies have been lacking and I can follow-up.
I noticed that you thanked another poster. If that's the type of post you've been expecting, I'm unable to meet your expectations. However, I can draw your attention to this person's claims that he pays no transaction costs, that "prime banks" (brokerages?) pay him rebates in FX, that he trades "interbank FX" (but doesn't know anything about the fee schedules on EBS - I edited that part of my post away in the thread below), that he has a "HFT operation (in FX, hosted at NY4)" (but doesn't seem to be trading on Currenex from all indications), and that he doesn't seem to know very well the difference between 0.19 cents and $0.19 (I corrected him but he made the same mistake twice).
This is a great thread Artemiso; even though the amount of 'Thanks' you've received seem to be inversely related to the quality of your replies. Nonetheless, thanks for starting this thread; it's a nice and insightful read, even though I’m not one of your target audience.
I'm a little late to this thread so I'm not sure if you still take questions, but if you do, I'd like to ask the following. Some of these questions are just me being curious and, also given your time, I’d respect your decision if you choose to discard these questions.
Questions about trading in general
1) At what time-frame do you think retail investors won't be affected by HFT? (i.e., above 1 minute chart, 5 minute)?
2) You've commented in this thread about how media and/or certain individuals frequently got it wrong (i.e., Irene Aldridge). What media and people do you find knowledgeable and thus worthy of reading?
3) MrYou suggests here that this is caused by algorithmic trading (which it probably is to an extent), but analysis from Nanex showed there was a 17 second lag between the tweet and the eMini reaction, suggesting that the initial reaction was caused by human traders.
What are your thoughts on events like this AP Twitter hack? How much do you see this caused by algorithmic trading and how much is this affected in your view by a lack of diversity among market participants? (Though these two factors will be related I suppose).
And, more generally speaking, what are your thoughts on using social media in trading given it’s complexities (i.e., high rate of false positives)?
4) I’m doubting if I should ask the following question since I don’t want suggest I’m disrespecting your decision to not talk about specifics, but since you’re catering here to a retail crowd on futures.io (formerly BMT), I wanted to ask you the following: can you comment somewhat on what you did when you were a retail trader?
Since in that situation you couldn’t use the expensive infrastructure and additional benefits (e.g. contacts with brokers, access to academic journals, attending conferences, having colleagues), how did you adapt? Did you used high time frames, or perhaps even just invested in ETFs instead of active day trading? And did you used retail platforms, or wrote your own?
I guess I’m trying to get an idea here how a professional would approach the limitations inherent into retail trading and, by getting a clearer view on this, also some implied guess on what you think are the odds of retail traders.
5) According to Aldridge (yeah I know.. ) (in High-Frequency Trading: A Practical Guide to Algorithmic Strategies and Trading Systems) it takes on average (!) 18 months to develop one successful HFT trading system and this is even with a small staff of people. What are your thoughts on this?
Questions about your work
1) Can you say some general comments about what people at a firm similar to yours are tasked with? For simplicity sake, let’s say 10 people work at a firm similar to yours. I suppose then that there is at least one quantitative trader, two or three quantitative developers, perhaps one financial engineer, and the remainder will be quantitative researchers?
I’m not asking to get an idea of your specific firm, but more what the general consensus for this is in the industry and how a work environment at firms like yours look like.
2) What does your typical/average day look like? Again, I do not mean to probe you for the specifics, but more the types of activities that you do. For example, is 40% of your time spend programming, 10% reading scientific papers, 30% doing mathematics, and the remaining time spend doing miscellaneous things?
3) What do you think are the 3 most important skills, besides mathematical and programming skills, for quantitative traders/researchers? In other words, which three skills are a basic requirement for anyone on your team?
4) Related to a question about programming I saw on SE: Do you believe that successful quantitative trading can be learned by anyone, or that some people ‘just don’t get it’? Does this depend, in your view, on general intelligence but also on personality and perhaps other factors?
Personal questions 1) This is also related to the team you work with, but how do you generate the (in hindsight) best ideas: do you believe these ideas are generated by a single person, or do good ideas in your view result from several people working together?
2) In my impression, you’ve been quite successful. Looking back at where you came from (i.e., before you attended university) which factors have, in your view, contributed the most to your success?
3) What are, in hindsight, some things that you’d have done differently if you were starting over from scratch? I’m not referring to things like ‘I shouldn’t have done trade XYZ or deployed model ABC’, but more general lessons you’ve picked up along the way (like the thought you offered in this thread that quite some people in your industry seem knowledgeable, or even have claimed to work somewhere, but are in fact full of sh*t).
Thanks for offering us the opportunity to ask you these types of questions Artemiso.
Wow those are excellent questions, of course I'd answer them.
I sort of understand. People want to hear about the latest exploits of de Rham cohomology and the Baire category theorem in the low-latency space. But if you come out and claim it's all about hard work and a sound understanding of fundamentals, very few want to pay attention.
I'll reiterate what I mentioned to MrYou:
- Since prices are already extremely efficient to news, it's not apparent to me how parsing news in social media as a signal to bid up the prices and walk the book is different from a discretionary, fat-fingered trade.
- Bloomberg already has machine-readable news and I admit having played around with it a little, to no avail. Your mileage may vary.
- I think what's interesting though is that these APIs and the dissemination of information can be used for much more meaningful, and potentially profitable means. A problem we faced at one of my previous workplaces was that the IRS was publicizing full scans of NPOs' disclosures and these included enough detail of the signatories to carry out a wide scale, automated wire fraud of our clients. It's scary what the domain of 'machine-readable' entails.
Conferences were still accessible. I attended seminars in my area as much as possible. I played around with a few retail platforms that I was exposed to since my university days (IB TWS, NT - I'm not that old) and also from some of my professional colleagues. A friend recommended I take a look at OpenQuant, Marketcetera and Tradelink and I didn't like them either. I spent maybe 1-2 weeks each - you might have seen some of my critiques on the forum - it was pretty quick to spot the problems if you've developed your own. A platform that doesn't teach you anything isn't worth learning. In that respect, I think all of them delivered, so it was time spent worthwhile.
I don't think the tools shifted my preference. I really think I would have built the same kind of strategies if I was more familiar with the .NET class library (I'm better with Fortran/MATLAB). I'll risk a limb here and say it's not that you can't do anything you wanted with NinjaTrader - I'm saying you can also use a key as a screwdriver but it won't be the most effective way to do it. I'll also be clear that there are limitations at the asymptotes: I benchmarked and it didn't make sense to me to try anything beyond a few symbols on second data. Don't quote me on it - do your own testing. There were also noncompete restrictions for most part so I had to trade in longer horizons. The same skills were highly applicable to measuring off-balance sheet A/L and I did a small amount of PE.
The main restrictions for a retail trader that I see are competitive commissions (someone on my team eventually negotiated down to 0.1 cps on some symbols), followed by knowledge, and capex.
Some think it's weird that I place knowledge before capex - there are very well-funded but unskilled speculators. Another demographic trend is that technology is catching up faster than the infrastructure requirements. For instance, I know BATS has no plans in its roadmap to provision for more than a 10 GbE connection at this stage.
Haha. Her estimate sounds reasonable. I'd put it in the order of 12 months with an experienced team. It gets more competitive with time - so one has to compensate for it with more experience or more hands - so I predict an aggregate behavior on this industry is that we are in a consolidation phase - it's been a poor year for finance jobs but comparably a good year for employment at the top 10 names (in my book).
I confess that I've only worked at large firms before this - which will not be a good sample because the work life and culture is extremely different from a typical Chicago prop firm that you'd probably be more interested to hear about.
At my previous workplace, the average person gets in around 6.45 AM, we prepare our daily brief before the opening. After that I try to debug the things I was working on that have dependencies, then it's lunchtime, and my afternoons are unstructured. I encouraged a pretty productive simulation cycle so throughout the day I'd be getting results. We had large and fragmented teams - you usually left one another alone, and the IT guys had absolutely zero idea what the quants did, though both groups were proficient programmers. I tried to run my team in a better way, although I still spoke to some guys more often via email than in person, even though we sat at talking distance apart. (If you're in a big firm, you will have a few models that only a few guys really understood but are used firmwide. If you're lucky, there's a clean, BIOS-looking GUI that you just need to shoot an email to the right department and you'd have access to in under an hour. On our floor, I received plenty of feedback that the limiting factor in the organization was the options valuation guy. He was one busy man. And you usually tried to leave him alone until late afternoon so you can get the data you want for your research, in batch. You tried not to bother him again even if you got it wrong. I think this was unproductive because it interrupted the strategy production cycle, but I left before I could improve the situation.)
If you were trading, you stayed until 6 PM+. If you were a research analyst, you stayed until the last metro (11PM~) or just stayed. Weekend and holiday overtime was well-compensated. There are guys who partied and had female company seemingly every night but performed like automatons in the day. I later became more familiar with the SV startup culture and it seems that this is as prevalent there, so the models-and-bottles association isn't very justified. At our floor, the fraction of happy marriages with kids, with growing seniority, was actually above average compared to other industries.
Hazing and budget immoderation still exists all across the industry. I didn't tolerate this from my team so we were good. But it seems that the ones that practise these always blow up, so there is a good karma at work. I think JPM still hosts its intern candidates at Four Seasons... just a speculation.
On Fridays, people seem to notice less if you leave at 3.45~ PM and if there's nothing much going on, your immediate bosses leave (a sign for the subordinates). I wonder if this has a behavioral effect on prices... In a larger, not-purely-quantitative firm, you have group analysts, micro analysts, active portfolio managers and "final" portfolio managers. Analysts work with the managers and the managers work with the "final" manager. Control gets in everyone's way besides the analysts. Generally your department reports both to senior management and the board. The actual work titles will vary with location.
My actual workload distribution has varied a lot over the course of this past year. It's hard to stick to a rigid schedule when you're just starting up. But it has some kind of pattern lately. Face-to-face interactions take about 40% of my time - travel time factored in - so far they've been more out-of-office than in-office meetings. Remaining 60% of the time is spent mostly on research. I do not have a structured daily routine for papers or programming, that usually varies with the 'new project' I'm pushing to production, but I do have a daily slot allocated to debugging.
My cofounders do a lot of the admin work that would traditionally have been my job, otherwise I'd estimate some 70% in meetings. Either ways 40~70% is a large number - so you better be good at what you're doing when you start, because there's very little time for self-improvement or research.
Hm. In practice: Usually, the choices are very obvious even though you have several candidates for the same job, and most of my employees were known through referral, so I don't think I had to think about 3 additional skills unless we needed someone domain-specific. Probably quick problem solving skills, if you consider that it as disparate from mathematical aptitude - brainteasers are common here but then again I'm not good at them myself. I like quick problems that test modeling skills and Fermi estimates (you recall the gravitational constant, number of days in a year, made some assumptions, and estimated how many He-4 particles were in the Sun).
Once you cross a certain threshold for aptitude, it is all fit. It's actually fine if you come in hoping to make a lot of money and tell us that your passion is making money, but you better have a good Fermi estimate for how much money you'd be making.
I think there is absolutely no innate element to quantitative trading. Most of it is unintuitive to the natural human state of mind so it has to be learned. The problem I see with the majority of retail speculators is that they spend most of their time looking for shortcuts in this learning process.
Definitely several people working together. One advantage which might be overlooked: In my experience, debugging is one area where multiple people can make faster-than linear gains. We practise a hallway testing culture where you have to pull someone unrelated to the project over to spot glitches.
Thanks. I don't consider myself successful, however, I think what have been most instrumental in my learning process were:
1. Most of my learning habits from elementary school and the first 2 years of middle-school.
2. Having excellent research mentors when I was in high school. Two quotes that stayed with me:
-- I was worried that I didn't know enough to embark on a project, so I asked if I should sit in on one of his college courses. He said it would be a waste of time for me to travel to the university just for one class, "I had to learn it (the material) on my own too." At some point in your life, there's no one else out there who can teach you what you need to learn - might not be today, but doesn't hurt to start learning on your own.
-- After I opened up to my next mentor - an extremely caring and thoughtful person - on many personal issues. "Such is life," was all he said, before moving on immediately to electroweak interactions. No point dwelling too long on the inconcrete.
3. Having a family that gave me autonomy but also support.
4. Tenacity, and a passion for puzzle-solving that I picked up in competitive math and chess.
5. Having a terrible high school that hardly cared, but also overachieving peers to compete against. You are approximately the arithmetic mean of the 5 persons you spend most of your time with.
There are many, and instead of giving an in-depth response that would only be applicable to someone who has to go through exactly everything I did, I'll give a response catering a variety of circumstances.
1. Starting a fund: I should have stayed longer at an established firm before going out on my own. When you think you're ready, wait another year (I very rarely take lessons from a discretionary trader but I admit this idea was reinforced by Martin Schwartz's autobiography).
2. Starting any kind of business: Everyone wants to know what you're doing and become a part of it. Be selective and don't waste time. Networking is highly overrated before you have a working product or are 1 month away from it.
3. Modeling or trading: Not really. Tail risk is overrated. Admit quickly if you only got lucky.
Also, I read this recently and thought it was very relevant to this topic. The reason that most retail speculators fail is that they aim to make money on the onset, not create wealth.
Wow, thanks for that elaborate answer Artemiso. My response is somewhat late, but this is not meant as a sign of disrespect or a lack of interest.
While your response was clear on many points, I have some follow-up questions and some new ones if you don’t mind:
1.1) I noticed that in other posts in this thread you also mentioned your social contacts in the industry, and you also mentioned in your response that a substantive amount of your time is spend with meeting people. Is this typical for someone working at an quantitative fund, or is this something that is primarily the result of your position?
1.2) Related to this, do you think that socializing and seminars have contributed to your trading results bottom-line? Do you think that retail traders, who often don’t have these resources, are missing out on something and should more actively pursue these?
2.1) It surprises me somewhat that you place commissions before knowledge. Obviously, commissions for a HFT retail trader – which I think you’re referring to here – are a huge cost of doing business. But that you place knowledge second seems to imply to me that the amount of knowledge needed might be less than it suggests at face value, since the largest restriction are the commissions and not (lack of) knowledge.
In an earlier thread you commented on having read just a few books when you left academia for quantitative trading, and picked the rest up in your on-the-job training. Do you think knowledge is overrated relative to skills?
2.2) When a HFT firm hires someone, will this person be put to a structured training to learn about quantitative trading and how the (software) environment of that firm works, or will a new employee be ready to work from the start?
3) This is an interesting point since it is related to the work/life-balance and how this (might) affect work performance. What are your thoughts on these points? Would you term yourself a workaholic or would you say part of your success can be attributed in striving to keep some ‘life’ besides work?
(Btw, thanks for your detailed description about the work. It sounds like a nice, stimulating environment to work in. )
4) That’s good news. Can you comment somewhat on the most detrimental shortcuts used by retail traders in your view? And what is your philosophy for learning this topic? I agree with you that taking shortcuts is, in the end, a waste of time, but on the other hand and speaking from personal experience, creating a good, structured curriculum when one is on the outside-looking-in is quite hard.
5) This is a interesting point, thanks. Since I suppose you’ve spend your fair share of time debugging over the years, do you have some general guidelines that might give us an idea as to how debug more efficiently?
6) That is a good point (and a great quote), since I wouldn’t be surprised if this aiming for money, instead of wealth, also leads to more shortcut taking and risky behaviours. But I wonder, in what way (or ways) can retail traders create value in your view?
artemiso, I wanted to say thank you very much for this thread; it is clear that you are an expert in your field and it's great to have you commenting on this topic. Cheers mate!
Yes, you have to have great people skill as a quant in finance, and you have to talk to plenty of people, often involuntarily. As a result of my position now, I have a choice not to talk to people, but I do so voluntarily: I think it's easier to build a successful business this way.
I've never thought of this question before. I think it's naturally beneficial if you spend more time with those who have parts of what you want to be. For this reason, I also don't play golf or go wine-tasting...
No, I'm not only referring to retail traders hoping to trade in short time scales. I'm referring to anyone who wants to trade; even large institutions. Even for daily strategies, I've seen loss of 96% the expected return just from an additional 5 basis points in transaction costs.
The knowledge is a prerequisite and don't get me wrong, there's a lot - but you can learn the content from scratch much quicker if you've been educated in a scientific discipline.
Yes, generally for 3 months. During this 3 months, they will also train you for the license exams you may need to pass. Before that, the firm would often recommend a book or two to read before you begin your work. In these firms, the turnover rate is very high and your first year's salary is usually substantially lower than what you would get at an investment bank (which you would have had a good shot of joining if you could get hired into HFT, although the skill sets are quite different), so getting past training is clearly insufficient to guarantee you success.
I wouldn't call myself successful but I think this is one of the explanatory variables that have a noisy effect. I've seen people who absolutely hate the line of work but I'd consider them pretty successful. I must say the majority of sell-side traders in my experience were superb at their work, but the moment they get off work, do not want to think about their work at all. On the other hand, I can't imagine doing anything else. Very often I'm out for a dinner ball or something or a gathering with old friends but my mind always drifts off back to work and I can't wait to get to a terminal or piece of paper. I realize there's always interesting questions to be asked and if once I've run out of interesting questions to ask, I would go back to school.
Don't know if I'm answering your question properly, but essentially, I don't think you have to strive to keep a certain threshold of life.
Thanks! I have to add that it's a good feeling but also a weighted sense of responsibility when someone with excellent credentials joins your firm, I take it as an honor to work with them more than that I'm conferring them the honor. e.g. I have a guy who was with Google for a long time, and I try very hard to make sure he gets the perks of his previous workplace at the current one.
I think the most detrimental shortcut is doing what Harvard is doing now. For a while now, Harvard's management company has been consistently delivering stellar returns away from the limelight of the fund management world. I think they returned about 14.8% last year, and they have a AAA rating. The problem is that as a consequence of this, they've been planning very optimistic expenditures on their budget, based off a 20% per annum return. I think the most detrimental shortcut is assuming that you will make a particular number.
Caveat: I don't come from a software development background. If I had to say, the best I can suggest is to get practice, but that's stating the obvious. However, keeping good documentation seems to be something you can do about it. Also, in my experience, programming is an exceptional area where multitasking is a bad idea. It's very tempting to finish up the core functionality and leave things incomplete or unoptimized until later on, but that becomes a huge source of debugging pain later on.
I can't suggest ways to create value as a retail speculator, but I can suggest ways not to create/encourage negative externalities at least: this includes doing one's due diligence and not buying books that should have been burnt, useless software packages and add-on indicators and too many screens when one doesn't need it.
Seems that Irene Aldridge has a really bad name, if the reviews on Amazon and her angry responses are anything to go by. And noted some comments in this thread too.
I have been waiting on reviews for her 2nd edition and first one today points out errors in the Google books sample and then gives 1 star for the whole book. Something really stinks here! Reviews should be on whole book not sniping on a single topic - programming languages.
For a lay person who does not need 100% technical accuracy like many critics want, do you think her 2nd edition book is at least useful for us wanting to get a broad understanding on HFTs? For $47 we should not expect too much really, given that the HFT world is so secretive. How much of Irene's flak is from those in the biz who object to making things more transparent I wonder? For that she does earn some credits from me. But should I buy the book?! The Kindle sample looks fine, but need a hardcopy for such a complex topic.
Aldridge doesn't deserve your $47, really. I just read through the entire Google Books sample and it is clear that it is still the same unoriginal content that was mashed together from a combination of Google and her LinkedIn contributors. There's no conspiracy to make the topic any less transparent to outsiders, except it reflects poorly on the rest of us when someone tries to represent your field and does a poor job.
Nonetheless, thanks for starting this thread; it's a nice and insightful read, even though I’m not one of your target audience.
