Experimental Flight Plans - Information

[charlymorton]

I am currently working on a system to automatically create flight plans based on tracking.

While this system is under development, I will release flight plans created by this system in this board.

Current development status - 21-AUG-2022
Multi-week only
Aircraft range not taken into account
Airport suitability check is almost fully implemented
No information on available repaints
Limited tracking sources
Flight plans use the normal download engine. This mean that any updates to flight plans will happen in the background.
Automatic publishing of flight plans
Flight plans will be published into to separate boards per country. The country will reflect the home country of the operator, not necessarily the country the aircraft is registered in.
See list below for supported countries

Planned development
Multi-week and single week flight plans
Add stops if leg is longer than aircraft's range
Add support for US, Manx, Jersey, Guernsey and Slovenian aircraft, with possibly more to come
Limited support for Latvian, Lithuanian, Spanish, Dutch and Danish registered aircraft.
Info on repaints with focus on painted aircraft with tracking
More tracking sources

Fully supported countries
Australia
Austria
Bahamas
Belgium
Bosnia and Herzegovina
Canada
Croatia
Czech Republic
Estonia
France
Hungary
Iceland
Ireland
New Zealand
North Macedonia
Norway
Slovakia
South Korea
Sweden
Switzerland
United Kingdom

[jhaley101]

Neat, thanks for sharing Morten

[mage]

Hi Morten,

If you want to use the airports/runways data held in my 52-week flightplanner as a basis for your later functionality (finding suitable airports based on runway length and surface) feel free to do that. It came from an open source cache of data that is more recent than FS9s, so although the planner is geared to FS9, that data might be suitable for your purposes. There were errors and omissions in the open source stuff - a lot of which I have corrected myself - but it might still be suitable. The open source data reflectes recent - post-FS9 - changes in  ICAO codes, so things like the re-coding that Malaysia and Indonesia have done are reflected in that data.

I also have an algorithm for looking ahead for destinations within a certain number of degrees and at certain ranges (intended to get aicraft home via intermediate airports rather than relying on random airport selection - e.g. using Beijing to get from Geneva to New Zealand along a Great Circle using the Haversine Formula), and a much shorter range version of that approach might be what you need to find a likely destination for a tracked aircraft. Often the destination can be worked out from the following departure.

The flightplanner is over on Flightsim.com, and although the coding won't be of much use, the underlying data may well be.

Good luck with the project, even an old FS9er like me is interested in what comes up!

[charlymorton]

Thanks Mage,

I would appreciate if you could send over the data you have, and the algorithm as well.

I used runway info from ourairports.com in my old flight planning system and I did some ICAO changes for the US, Canada, Malaysia and Indonesia to fit with FSX, but I haven't ported the data over to the new system yet and I don't know how good the quality of my data is.

Cheers,
Morten

[mage]

Hi Charly -

The data is all in the flightplanner I uploaded to Flightsim.com, at https://www.flightsim.com/vbfs/fslib.php?do=copyright&fid=214972 where there are 5 example spreadsheets that all contain the data. It can be stripped out of the Excel tabs and used in text form, or left in the spreadsheet.

Since you want the "algorithm" - which is actually a large VBA macro - it's best to be in the spreadsheet to look around. Obviously you would need access to MS Excel or another program that can read spreadsheets created in Excel 2007 (the version I used).

The data isn't changed by the algorithm, it's used to verify or autofill other data fields and as a read-only source for the algorithm.

The spreadsheet is for FS9, as already said, but FSX and later airport lists could be used to generate plans, although the database might then need some adaptation for new airports. There were many airports from the Open Source website that had aspects missing such as runway surface type, which is important for my generator program (built on a basis of the RPP flightplan generator that Rysiek was fine with me adapting for my own purposes and for releasing). If a runway type is missing, aircraft expecting to land on hard runways would not visit, for example. So the planner needs to know runway lengths, aircraft performance (takeoff distance) to see whether it could land/depart the airport. I've done a lot to ensure that the Open Source data is improved for the planner, but there will still be holes in the data, for example in the Australian outback and in China.

The VBA code is not as nice as I would like. I developed from Rysiek's code for a small project that I completed 2 years ago (!), stripping his program down to do just what I wanted, and then I had an idea for a new project that meant building it back up and expanding it hugely (52-week flightplans, of which I have about 400 operators in my setup now).

If I had sat down and designed it properly rather than discovering ideas as I went, it would have been neater.

It is basically a random flightplan generator that gives us a lot of options to take the randomness away, such as favoring countries and denying access to others, weighting countries as to how likely they are to be visited, and also subtle factors like "the further an aircraft flies, the larger the airport it must use on arrival" (done by artificially shortening the runway length in accordance to distance flown). This stops a bizjet from Munich arriving at a deserted airport in the USA that just happens to have a long enough runway. So there are lots of ideas hidden in there.

It also does TNG flights, and since it can be used to make plans for large fleets (or even multiple fleets in the same batch) there is a system for TNG where you give each individual aircraft a percentage chance of asking for a TNG detail, and an airport is given a percentage chance of accepting the request. If the aircraft gets a random percentage below the threshold (for example 25% - once in four flights it will request to fly TNGs) it will be asking to fly some TNGs, if the airport has a TNG percentage of 50% it will accept TNG requests 50% of the time. So if the aircraft percentage in this case was 12% and the airport random number was 75%, the aircraft would request TNGs but the airport would "refuse". Balancing these two thresholds can get you a realistic level of TNG activity for FTOs (flight training organizations). You might want an operator of Cessnas to have C150s performing TNG trips 70% of the time, C172s performing TNGs 15% of the time, and C182s performing TNGS only 3% of the flights that they make. You can then select the airports where they will perform these TNGS by opening up the TNG percentage from zero (closed to TNGs - default) to some value such as 33%, accepting TNGs 1/3 of the time.

Entire airport categories can also be given the option of accepting TNGs, although this is a bad idea for large airports! For example all small airfields could be given a global percentage chance of accepting TNGs of perhaps 25%, but you only allow TNGs at a few selected medium airports. For example in England there is TNG traffic allowed at places like Exeter, Bournemouth, Cardiff, Oxford, and Biggin Hill, all of which are quite busy airports, some with commercial services. So you could set all small airports to allow TNGs 50% of the time, and these larger airports individually have values such as 25% or 10%, so that they get TNG flights less often.

The whole thing about this planner is that the destinations are randomly chosen, but are then rejected according to the values we put into the controlling factors, such as runway length, or country status (nobody wants to fly to Iran in general, and Cessna 150s are not generally allowed into large airports like Heathrow). What we get eventually is a set of hopefully plausible destinations and a more realistic aircraft behavior. For example regarding behavor of the AI, you can increase or lower a "base factor" that extends the aircraft's sit-time on the ground if it is at its base. If it lands away it might be allowed to sit there for 100 minutes, but at base you can make it sit on the ground for longer times. This is very useful for Air Ambulance operations, where the aircraft sits at base for a long time and then gets called out to a destination where it makes a very fast turn-around. You set the maximum sit-time to 30 minutes and then set a base bias of 6, and when it gets back to base it will potentially sit there for 3 hours (30 minutes x 6). Other aircraft will also tend to rest at base for longer times than at destinations.

The planner also copes with timezones. Each AI can use a profile for flying that dictates the hours it is allowed to fly in each day. This was devised by Rysiek Winawer and is a great idea. The problems are that FS9 flightplans are using GMT/UTC when compiled but local times are when we want the AI to operate. In the planner, I've made the AI "aware of where it is" and slip the timings to reflect this from a timezone database that I created from various bits of Wiki data and a lot of hard work! In countries with multiple timezones but no way of telling which one an airport is in (USA and Canada, and China, that uses one official timezone but actually spans five), I use longitude instead (1hr = 15 degrees of longitude) to make the AI land and depart within reasonable hours (as specified in the profile that the aircraft uses). So if an AI bizjet departs Chicago on a flight that will see it arrive in Bratislava at 2am, the planner will slip the departure on by an hour until the arrival is also within a correct local time in Bratislava. This reflects the reality of late departures from the USA arriving into Europe early in the morning and landing after curfews are lifted.

I have attached some 52-week plans made in the last 24 hours so that you can see the output.

There is a value on the MyProject page (at the upper right in a hidden field) that governs how many flightplans are created. If you clear this from the default value of 52 (weeks) it will create a single week's flightplan. When I was making a package for Air Force One (USA) I set the planner to create several hundred flight plans so tha tI could extract suitable flights from a large numebr and stitch them together to make better weeks. Air Force One flies with an entourage of other aircraft and the planner won't allow me too much discretion in getting an entire fleet to travel around in a cohesive group to a variety of airports close to the destination. Air Force One might fly into Heathrow, but the support aircraft are likely to go into Farnborough, Northolt, and Stansted. I can't really automate that!

Hopefully you can at least use the data. The algorithm might have some useful features in there that can be used, but it is Open Source, and anything built from it would also need to be Open Source.

[charlymorton]

Thanks Mage,

If I had sat down and designed it properly rather than discovering ideas as I went, it would have been neater.

I can definitely relate to this, reading your post makes me realise I need to add support for TNGs. Additionally, my code is a mess and poorly commented, so I spend ages trying to figure out why I did something a certain way, especially if I haven't worked on that part of a code for a while.

I actually used Rysiek's spreadsheet a long time ago and remember I made some changes to suit my setup better.

Anyway, the way my old setup worked, was that I collected aircraft information manually from available sources, assigned them to the correct operator in my spreadsheets and collected tracking data manually (I later upgraded this to run a PowerShell script to download tracking).
The tracking data was put into a spreadsheet where a bunch of formulas were used to convert times from local time zones to UTC, check the departure airport matched the arrival airport of the previous leg, check the distance travelled could be covered by the aircraft and check the time of the leg matched what was feasible. Any errors had to be overridden manually. The original data was for multiweek flight plans, however I also created formulas to find the busiest week per operator and move those data to a weekly flight plan for FS9 users.
Again, this part was later mainly automated by a PowerShell script, which also included a check of the runway suitability.

The problem with this approach has always been that it's been extremely time consuming, even though parts are automated.

With the new approach I am planning to fully automate "everything". Originally I planned to create one massive script to do it all, however I've checked it to have one script per country to collect aircraft information and one script to create flight plans for all collected aircraft information.

For example for Norway, the script will download the main aircraft register, which is a spreadsheet made available twice year, before downloading the monthly changes spreadsheets.

It will then run through the main spreadsheets, and then update any aircraft based on the monthly changes by deleting, adding and changing operators as needed.

The next step is to check my setup for flight sim to see if any of the aircraft are assigned to a different operator so that I can ensure my own setup will match the data in the flight plans I create.

It will then remove any aircraft I don't care about, such as helicopters, balloons, and anonymises any aircraft registered to individuals. It also removes any aircraft operated by larger airlines, as there is no point releasing flight plans for them, since AIG and others do, and adding the ones I do want tracking for to a shared text file.

When running the flight planning script, it will check that I have correct info (cruise speed, ceiling, range, runway requirements and wing span) for all the aircraft in the shared text file.

If good to go, the script groups the aircraft based on operator and wing span, downloads tracking, converts it to FS format, zips the files up.

Apart from some code hardening to check airport suitability, leg distance and conversion to weekly flight plans, my next step is to make it automatically upload and publish the flight plans to the website.

Also, I want to have support for as many countries as possible.
Currently I have full support for Norway and Sweden. Denmark is on hold until I can verify operators.
Full support for New Zealand is almost done, so will be next. Support for the UK is in progress.
Will start working on Ireland, Slovenia and Spain soon.

With each new country there is a lot of data to verify manually, however when it has been done once, updates should be much less time consuming.

[mage]

Funny that you are not interested in helicopters. It was UK helicopters that got me interested in adapting Rysiek's spreadsheet. It is now mutated beyond all recognition. I was interested in putting the many AI helicopters available as FS9 models into the sim (it was to be my final large project for flightsim before I "retired" completely). I fiddled with flightplanning and getting the models to behave properly and then researched all the aircraft that could be injected into the sim, and at the time there were 808 of them. Making a flightplan for each of 808 aircraft seemed crazy and I figured it would be quicker and less error-prone to use Rysiek's software. Except that the RPP planner has all the aircraft based at one airfield and plans outward from there, and I wanted 808 aircraft all based in different places. So the job began! I stripped out code I didn't need and made this simpler version deal with multiple bases and made the plans.

But I was making 52-week plans by hand since 2009 and realized that this would make that other process much simpler, so I added lots of things back in as well as lots of new ideas and it grew from there. I used to take all day to generate 52-weeks and check through the plans (Ben Laenens and other random generators). I can now put in the aircraft details into a spreadsheet and run off 52 plans in 10 minutes - 9 minutes of which is preparing the aircraft data and pressing the button!

The only shortcut I adopted was to make a generic spreadsheet version for each continent, there are many airports in the Americas and long-range aircraft based in Europe would have a tendency to flock over to the USA and fly there simply because there are more airports to randomly choose. So a European version of the spreadsheet will boost European airports in the rankings and drop the US airports down the rankings a little (a lower percentage chance of being selected), and eventually I will get sets of plans that are broadly right in how often that Gulfstream 5 will cross the Atlantic, etc. Once I had one that was "balanced" properly I can just use a copy of that for each European operator without having to balance the flying out anymore. As a result I can get an operator up and running quite quickly.

Most of the European-based repaints you have here are now planned up for my sim! Danny has just made some new ones and I have a bit more work to do!

These plans will be released on Flightsim fairly soon. The first set released (about 80 plans) was for fleets that are only partially repainted, and this time around I have concentrated only on aircraft that are actually repainted (leaving other parts of the fleet out if there are no paints) with another 300 sets of plans. This has sped up the installations a huge amount since I am no longer waiting for repaints to show up. If a repaint appears I can very easily add that aircraft to the planner and create another set.

The only other things I leave out are plans for AI such as the DFAI models. I don't have these and as a result make plans only for the freeware models that most people will be using. If they were in my setup I would obviously include them in my sets of plans because it would be mad not to!

I appreciate that it is more accurate to replicate real world routes etc, if that is what interests people, but I fly everywhere in the sim and all I really need is believable traffic at the places I visit, not necessarily accurate traffic! The idea that it can change every week is a bonus.

[charlymorton]

I like the idea of having different plans per week. For me having the possibility to use multi-week plans does that for me.

I ran my script now to create flight plans for Norway, Sweden and New Zealand and it ended up taking 41 hours to complete and generated flight plans for 670 operators + one flight plan per country for privately owned aircraft.
The largest one was the flight plan for privately owned aircraft in New Zealand where tracking was found for 370 out of 1400+ registered aircraft and that took eight hours to generate.

The code for this could definitely be optimised to improve the speed but some of the slowness is due to my laptop starting to show its age.

The next step is to correct any bugs and add runway info for airports where this is missing in my data, before I am running this again with UK aircraft added to the list.

In my next run, which I hope to start tonight I am planning to have automatic publishing added, so if you see a lot of posts being added to the website it will be done by the script.

[charlymorton]

I had to do a complete re-run of the flight planning script due to some critical changes.
Unfortunately I made a mistake when I started the run, which lead to duplicate posts.
In the next days I am removing all the posts in this board and creating new posts in country specific sub-boards. I am also fully supporting French registered aircraft, which has been a complete pain to do, but the brunt work has been done at least

[mage]

That's a nuisance, although I met the same situation with the Excel planner, creating some important new feature (such as timezones, and then timezone slipping - done as two separate improvements). After each one, the effect was so useful that I felt compelled to re-make everything that had already been created, which at the time was about 60 operators! So I feel that pain!

Does your processing allow you to make a good guess about aircraft bases? That's the one greatest time-consuming part of making the plans that I'm doing.